Category Archives: Programming

Initial release of the i18n branch of osm rails-port

Now, we have a working i18n branch for the osm rails_port. This setup is done by using globalize and click-to-globalize plugins. As of now all/most strings of website can be translated using the present translation interface. We even have l10n statistics, translation update feeds and more…

SVN Branch: [http://svn.openstreetmap.org/sites/rails_port_branches/i18n/]

Wikipage: [http://tinyurl.com/osm-i18n]

More detailed writeup: [http://makghosh.googlepages.com/gsoc-writeup.pdf]

Loads of thanks to my mentor Mikel Maron and the entire osm community for guiding and helping me out in internationalizing the rails-port :)

Another major chunk of work that will follow this is, map-tile localization. Also there are some enhancements to be done, listed in todo section of the wikipage. So discussions are open. And, i’m really glad that I passed my final evaluation of Google SoC, thanks again goes to my mentor :) It was a rocking summer!!

Language filtering of diary entries!!

This feature was originally proposed by Dan Karran as a comment in one of my earlier posts. Anyway, so here we have “show all / filter by my language” tags in the user diaries’ pages. By default, all entries are shown. We can click on ‘filter by my language’ so as filter the posts by the locale selected by the user in his preferences. We can also have feeds for these filtered entries. The screen-shot of current diary entries page,

openstreetmap-language-filter-diary-entries

Image #1: openstreetmap-language-filter-diary-entries

And for this we need to specify the language of each diary entry. So new entry page has an option or rather a dropdown menu from where we select the language of the entry. This preference, by default takes the value of user’s current locale and is saved in language column of diary_entries table. The screen-shot of ‘new diary entry’ is as follows,

openstreetmap-new-diary-entry

Image #2: openstreetmap-new-diary-entry

Also put up your suggestions… :)

Localization Statistics

After translation updates, I did the l10n statistics. This basically give the translators an idea of number of pending/completed strings and percentage of localization done for all locales currently supported by the app.  For this i created a Statistics table in database using migration. Now the statistics are updated when user clicks on the l10n link on his/her home page. The logic (in TranslateController#stats) is if the current language of the user is not present in the statistics table, it is added. Otherwise normally the values are updated with the help of globalize_translations table. The screen-shot of the l10n statistics,

openstreetmap-l10n-statistics

Image #1: openstreetmap-l10n-statistics

Its’ the ‘l10n statistics’ that apears first when the translator clicks on l10n link in his/her home page. Another nifty feature is that, s/he can view the sort the l10n statistics in two ways, ‘by percentage’ or ‘by number of completed strings’ in descending order. Also, every locale has a link to the pending/completed strings page associated with the numbers.

Apart from this, i have kept this l10n statistics page (without those pending/completed strings link) visible to all users, just to motivate more of them into translation. The screenshot of user’s home page with l10n link (view rendered in bn-IN locale as you can see one string i translated for example),

openstreetmap-l10n-example

Image #2: openstreetmap-user-home-bengali-partial

The difference between user/translator is done by tr_status column in user table,

0 = User
1 = Translator
2 = l10n Admin (proposed)

Todo: For now, the tr_status is updated manually. We have thought about a admin page for the l10n admin where s/he can add/remove translators, add/remove coordinators etc. Thoughts??

Please put up your comments or suggestions…

Translation updates via RSS

Its’ been a while since i last posted an update. So now, I need to open-discuss all my exploits for last 2-3 weeks. I owe lot of thanks to my mentor Mikel Maron for helping  me out in testing different features, pointing out some important bugs, suggesting new features and how to code/implement them. Now, coming straight to the point, with help of globalize plugin in osm rails-port, we now have translation feeds. I mean now translators can now have rss feeds for their default language (say spanish (es-ES)), both for pending and completed strings. These are the strings generated or rather added in the globalize_translations table while different views are rendered in that locale.

We have the entire list of pending or completed strings for a particular locale (which is actually the language chosen in user settings page) paginated in groups of ten, which is in turn available as updates by rss feeds. We can simply move to “older strings / newer strings” accordingly. For doing these i needed to add translate_controller.rb, corresponding views and also updated the routes.rb as required. The screen-shot of the “l10n home” that we have in osm rails-port now,

openstreetmap-l10n-home-pending-strings

Image #1: openstreetmap-l10n-home-pending-strings-with-rss

Also, each string has a link that opens up the translation interface where we can view the string and add/update the translation for that particular string. For this purpose, I have used a form, where we can type-in the translation for the string in the translators’ locale. When we add/update translation for a string, its’ updated in the database and if everything works fine, we are confirmed by a flash notice “Translation was updated successfully”. The next time we open the corresponding view, we find the string translated. Here’s the screenshot of the translation interface,

openstreetmap-translation-interface

Image #2: openstreetmap-translation-interface

There are some languages which are common to multiple countries. So i had to update the views so that locale without any country tag can be added. I mean earlier, Spanish(es-ES) & German(de-DE) etc got to have the country tag. But now languages like Arabic(ar) common to multiple countries can also be added.

Todo: Now this web-based translation is actually meant to be handled by the ajax in-place-editor of click-to-globalize. Because it places the translatable strings in context, as an advantage. But the problems with CtG are partially fixed. More on this at a later post. So we need the in-place editor start working. Although, the existing translation interface will stay on, so that translators can update translation of any string on the go, if needed just by checking the feeds. They don’t have to search for the concerned view! Thoughts??

User’s language preference added in osm rails-port

In last couple of days, I added the user language preference in the osm rails-port. It is basically implemented by a drop-down menu in the settings page of the user, from where he can easily select & save the desired locale. When the selection is saved, it’s saved in the database where a locale column has been added to user table by migration (012_add_user_locale.rb).  The drop-down menu is implemented in the view (account.rhtml) by a helper method called “select” as given,

select ("user", "locale", { "English(US)" => "en-US", "Bengali(IN)" => "bn-IN", "Hindi(IN)" => "hn-IN", "Spanish" => "es-ES" })

Correspondingly, the user’s model & controller has been updated. Here, i have used a hash within the select method for specifying languages, which can be easily replaced by a hash variable defined in config/environment.rb for convinience. Another approach maybe to find all the locale codes from globalize_translation table migrated by the globalize plugin using “collection_select” helper method. But, in the previous one we actually will have the flexibility to add only those languages we prefer (it may be based on availabity of translations or rather request for the language branch by an l10n team).

This language preference option is somewhat independant of the plugin problems. But the locale-routing part is yet to be decided, regarding how to do that, because click-to-globalize also implements some sort of routing associated with locale_controller.rb. The screenshot of the user settings page:

User's language preference added!

User's language preference added.

More importantly, a sort of bug that came with the globalize plugin as i have posted earlier (error log) is now fixed. As it can seen in the log, while I tried to create a new user, <% error_messages_for 'user' %> is called in the app/view/user/new.rhtml. Now, globalize plugin overrides the “error_messages_for” helper method for it’s own requirements. The helper method tried to execute nil.errors? (from the error log) which is quite obvious, since the object user is created after rendering the view and resulted with errors. I first tried to take care of this by a small change in active_record_helper.rb within the globalize plugin:

return "" if object_name.nil?

But it didn’t do much good. Later i found a better fix from rails-forum which actually worked flawlessly. Here’s the new active_record_helper.rb. So one of the two big problems i previously blogged with these plugins is solved. Since, globalize started working i could also test the views related to ‘user’. There were some sillymistakes for example, forgot to use to_s in @user.messages.size in the globalized view.

"You have %s new messages and".t(nil,@user.new_messages.size.to_s) + " %s old messages".t(nil,(@user.messages.size - @user.new_messages.size).to_s)

I have commited all these updates in the i18n branch. Also, apart from the inherent loopholes in click-to-globalize, we are also focussing on other areas that needs to addressed including translation notifications etc. Please put up your comments and suggestions :)

Globalization of views done

Today i finished commiting all the globalized views. You check them out in the svn: [http://svn.openstreetmap.org/sites/rails_port_branches/i18n/app/views]. Some of the views have simply <%= "text to globalize".t -%> things and some also have <%= "Hello, %s".t(nil,@user.name) -%> sprinf() like stuff. This basically means that whenever translations are available in the database, they get rendered into the pages for that particular locale. But now, the next step is to add a mechanism to feed translations and change/preference of user’s locale, which we decided to be handled by click-to-globalize plugin.

Currently, it’s giving some serious problem like [http://makghosh.fedorapeople.org/osm/ctg-error.log]. I talked with svenfuchs in #rubyonrails to know that the latest globalize plugin has been moved to the github. Hope this will solve the problem somewhat. And also while creating a new user a nil object NoMethodError is caught by the exception. I’m looking into these for a fix ASAP. It would be great if someone can help me to sort this out.

Update 101: “The magic has begun”

All these days i was continuously working setting up the infrastructure ready, doing custom setups and testing things for the i18n of openstreetmap, more precisely setting up internationalization in rails which will comprise a major chunk of my gsoc project.

I started off by setting up an osm local install on my desktop earlier this month. That was something because there different set of things to look after and configure each of them properly. I got little stuck with the mod_tile thingy…the configs. I just couldn’t find where was some hard-coded paths renderd daemon was trying to access. Thanks to TomH & Alex, for mentioning about gen_tile.cpp. I thought all of these paths should be placed in render_config.h and even made a patch of it (to ease my work) but later learnt and/or realized that they are meant to serve some greater purpose like module options actually.

So, crossing all this, when rails together with mapnik powered on for the first time i was awestruck. I just wanted to laugh…whew!! Then after some discussions with my mentor Mikel Maron, we looked up at the different options available. He gave me a comparative study of different ways to internationalize rails. I kinda liked two amongst them namely, globalize-rails and ruby-gettext (which i had proposed earlier). So as planned, I somewhat did some quick hacky test setups to test both of them. Though they weren’t free of problems, but as of now, i simply like the robustness of globalize combined with click-to-globalize plugin. Also the the fact that globalize currently supports more languages give it a plus. Sven Fuchs has written some really cool tutorials….thanks to him because i don’t know for some reason whatsoever i’m not getting the www.globalize-rails.org. Anyway, the current state is i’m trying to setup globalize in the rails_port of osm now. Its’ bound to have lots’ of problem…need to fix them. Another concern is that about the translators’ accounts. Whether it will be wiki-like so that anyone can put up translations or the traditional translation groups somewhat moderated. Though this will come at a later stage.

In the meantime, i actually upgraded my desktop to an E8200 2.66 GHz proc. + DG33FB motherboard + 2 GB combination and its’ showing some real power. So, another ordeal that came up of nowhere was shifting the entire setup to the new hdd. It made me remember of the osm vmware image that Milo van der Linden talked about in the early stages of the project. There was some real hike in the rendering speed compared to my 5-yrs old celeron 1.7 GHz mobo.

But now, my 6th semester exams are scheduled from 3rd-13th of June and i’m really getting low working hours now :( I will also put up a wikipage regarding i18n sooner than later. Loads of work still left…

Among other news, my activity oriented tutorials are up on the edu.kde.org …lots of thanks to Anne Marie (annma) for putting them up. I just love the logo guy pointing to the tutorials. Hopefully, i’ll try to put more of them. And more recently Debayan from NIT-DGP has put up the 3rd ntp pool server for India (with my WBUT starting the spur as always). I can now three ntp servers in http://www.pool.ntp.org/zone/in.

Update: Also as a matter of fact, globalize-rails now supports much more languages compared to ruby-gettext. And when combined with click-to-globalize plugin, i can see that we can even actually chip in translations from a WebUI….which is pretty cool :)