Now, we have a working i18n branch for the osm rails_port. This setup is done by using globalize and click-to-globalize plugins. As of now all/most strings of website can be translated using the present translation interface. We even have l10n statistics, translation update feeds and more…

SVN Branch: [http://svn.openstreetmap.org/sites/rails_port_branches/i18n/]

Wikipage: [http://tinyurl.com/osm-i18n]

More detailed writeup: [http://makghosh.googlepages.com/gsoc-writeup.pdf]

Loads of thanks to my mentor Mikel Maron and the entire osm community for guiding and helping me out in internationalizing the rails-port

Another major chunk of work that will follow this is, map-tile localization. Also there are some enhancements to be done, listed in todo section of the wikipage. So discussions are open. And, i’m really glad that I passed my final evaluation of Google SoC, thanks again goes to my mentor It was a rocking summer!!

This feature was originally proposed by Dan Karran as a comment in one of my earlier posts. Anyway, so here we have “show all / filter by my language” tags in the user diaries’ pages. By default, all entries are shown. We can click on ‘filter by my language’ so as filter the posts by the locale selected by the user in his preferences. We can also have feeds for these filtered entries. The screen-shot of current diary entries page,

Image #1: openstreetmap-language-filter-diary-entries

And for this we need to specify the language of each diary entry. So new entry page has an option or rather a dropdown menu from where we select the language of the entry. This preference, by default takes the value of user’s current locale and is saved in language column of diary_entries table. The screen-shot of ‘new diary entry’ is as follows,

Image #2: openstreetmap-new-diary-entry

Also put up your suggestions…

After translation updates, I did the l10n statistics. This basically give the translators an idea of number of pending/completed strings and percentage of localization done for all locales currently supported by the app.  For this i created a Statistics table in database using migration. Now the statistics are updated when user clicks on the l10n link on his/her home page. The logic (in TranslateController#stats) is if the current language of the user is not present in the statistics table, it is added. Otherwise normally the values are updated with the help of globalize_translations table. The screen-shot of the l10n statistics,

Image #1: openstreetmap-l10n-statistics

Its’ the ‘l10n statistics’ that apears first when the translator clicks on l10n link in his/her home page. Another nifty feature is that, s/he can view the sort the l10n statistics in two ways, ‘by percentage’ or ‘by number of completed strings’ in descending order. Also, every locale has a link to the pending/completed strings page associated with the numbers.

Apart from this, i have kept this l10n statistics page (without those pending/completed strings link) visible to all users, just to motivate more of them into translation. The screenshot of user’s home page with l10n link (view rendered in bn-IN locale as you can see one string i translated for example),

Image #2: openstreetmap-user-home-bengali-partial

The difference between user/translator is done by tr_status column in user table,

0 = User
1 = Translator
2 = l10n Admin (proposed)

Todo: For now, the tr_status is updated manually. We have thought about a admin page for the l10n admin where s/he can add/remove translators, add/remove coordinators etc. Thoughts??

Please put up your comments or suggestions…

Its’ been a while since i last posted an update. So now, I need to open-discuss all my exploits for last 2-3 weeks. I owe lot of thanks to my mentor Mikel Maron for helping  me out in testing different features, pointing out some important bugs, suggesting new features and how to code/implement them. Now, coming straight to the point, with help of globalize plugin in osm rails-port, we now have translation feeds. I mean now translators can now have rss feeds for their default language (say spanish (es-ES)), both for pending and completed strings. These are the strings generated or rather added in the globalize_translations table while different views are rendered in that locale.

We have the entire list of pending or completed strings for a particular locale (which is actually the language chosen in user settings page) paginated in groups of ten, which is in turn available as updates by rss feeds. We can simply move to “older strings / newer strings” accordingly. For doing these i needed to add translate_controller.rb, corresponding views and also updated the routes.rb as required. The screen-shot of the “l10n home” that we have in osm rails-port now,

Image #1: openstreetmap-l10n-home-pending-strings-with-rss

Also, each string has a link that opens up the translation interface where we can view the string and add/update the translation for that particular string. For this purpose, I have used a form, where we can type-in the translation for the string in the translators’ locale. When we add/update translation for a string, its’ updated in the database and if everything works fine, we are confirmed by a flash notice “Translation was updated successfully”. The next time we open the corresponding view, we find the string translated. Here’s the screenshot of the translation interface,

Image #2: openstreetmap-translation-interface

There are some languages which are common to multiple countries. So i had to update the views so that locale without any country tag can be added. I mean earlier, Spanish(es-ES) & German(de-DE) etc got to have the country tag. But now languages like Arabic(ar) common to multiple countries can also be added.

Todo: Now this web-based translation is actually meant to be handled by the ajax in-place-editor of click-to-globalize. Because it places the translatable strings in context, as an advantage. But the problems with CtG are partially fixed. More on this at a later post. So we need the in-place editor start working. Although, the existing translation interface will stay on, so that translators can update translation of any string on the go, if needed just by checking the feeds. They don’t have to search for the concerned view! Thoughts??

In last couple of days, I added the user language preference in the osm rails-port. It is basically implemented by a drop-down menu in the settings page of the user, from where he can easily select & save the desired locale. When the selection is saved, it’s saved in the database where a locale column has been added to user table by migration (012_add_user_locale.rb).  The drop-down menu is implemented in the view (account.rhtml) by a helper method called “select” as given,

select ("user", "locale", { "English(US)" => "en-US", "Bengali(IN)" => "bn-IN", "Hindi(IN)" => "hn-IN", "Spanish" => "es-ES" })

Correspondingly, the user’s model & controller has been updated. Here, i have used a hash within the select method for specifying languages, which can be easily replaced by a hash variable defined in config/environment.rb for convinience. Another approach maybe to find all the locale codes from globalize_translation table migrated by the globalize plugin using “collection_select” helper method. But, in the previous one we actually will have the flexibility to add only those languages we prefer (it may be based on availabity of translations or rather request for the language branch by an l10n team).

This language preference option is somewhat independant of the plugin problems. But the locale-routing part is yet to be decided, regarding how to do that, because click-to-globalize also implements some sort of routing associated with locale_controller.rb. The screenshot of the user settings page:

User's language preference added.

More importantly, a sort of bug that came with the globalize plugin as i have posted earlier (error log) is now fixed. As it can seen in the log, while I tried to create a new user, <% error_messages_for 'user' %> is called in the app/view/user/new.rhtml. Now, globalize plugin overrides the “error_messages_for” helper method for it’s own requirements. The helper method tried to execute nil.errors? (from the error log) which is quite obvious, since the object user is created after rendering the view and resulted with errors. I first tried to take care of this by a small change in active_record_helper.rb within the globalize plugin:

return "" if object_name.nil?

But it didn’t do much good. Later i found a better fix from rails-forum which actually worked flawlessly. Here’s the new active_record_helper.rb. So one of the two big problems i previously blogged with these plugins is solved. Since, globalize started working i could also test the views related to ‘user’. There were some sillymistakes for example, forgot to use to_s in @user.messages.size in the globalized view.

"You have %s new messages and".t(nil,@user.new_messages.size.to_s) + " %s old messages".t(nil,(@user.messages.size - @user.new_messages.size).to_s)

I have commited all these updates in the i18n branch. Also, apart from the inherent loopholes in click-to-globalize, we are also focussing on other areas that needs to addressed including translation notifications etc. Please put up your comments and suggestions

Today i finished commiting all the globalized views. You check them out in the svn: [http://svn.openstreetmap.org/sites/rails_port_branches/i18n/app/views]. Some of the views have simply <%= "text to globalize".t -%> things and some also have <%= "Hello, %s".t(nil,@user.name) -%> sprinf() like stuff. This basically means that whenever translations are available in the database, they get rendered into the pages for that particular locale. But now, the next step is to add a mechanism to feed translations and change/preference of user’s locale, which we decided to be handled by click-to-globalize plugin.

Currently, it’s giving some serious problem like [http://makghosh.fedorapeople.org/osm/ctg-error.log]. I talked with svenfuchs in #rubyonrails to know that the latest globalize plugin has been moved to the github. Hope this will solve the problem somewhat. And also while creating a new user a nil object NoMethodError is caught by the exception. I’m looking into these for a fix ASAP. It would be great if someone can help me to sort this out.

All these days i was continuously working setting up the infrastructure ready, doing custom setups and testing things for the i18n of openstreetmap, more precisely setting up internationalization in rails which will comprise a major chunk of my gsoc project.

I started off by setting up an osm local install on my desktop earlier this month. That was something because there different set of things to look after and configure each of them properly. I got little stuck with the mod_tile thingy…the configs. I just couldn’t find where was some hard-coded paths renderd daemon was trying to access. Thanks to TomH & Alex, for mentioning about gen_tile.cpp. I thought all of these paths should be placed in render_config.h and even made a patch of it (to ease my work) but later learnt and/or realized that they are meant to serve some greater purpose like module options actually.

So, crossing all this, when rails together with mapnik powered on for the first time i was awestruck. I just wanted to laugh…whew!! Then after some discussions with my mentor Mikel Maron, we looked up at the different options available. He gave me a comparative study of different ways to internationalize rails. I kinda liked two amongst them namely, globalize-rails and ruby-gettext (which i had proposed earlier). So as planned, I somewhat did some quick hacky test setups to test both of them. Though they weren’t free of problems, but as of now, i simply like the robustness of globalize combined with click-to-globalize plugin. Also the the fact that globalize currently supports more languages give it a plus. Sven Fuchs has written some really cool tutorials….thanks to him because i don’t know for some reason whatsoever i’m not getting the www.globalize-rails.org. Anyway, the current state is i’m trying to setup globalize in the rails_port of osm now. Its’ bound to have lots’ of problem…need to fix them. Another concern is that about the translators’ accounts. Whether it will be wiki-like so that anyone can put up translations or the traditional translation groups somewhat moderated. Though this will come at a later stage.

In the meantime, i actually upgraded my desktop to an E8200 2.66 GHz proc. + DG33FB motherboard + 2 GB combination and its’ showing some real power. So, another ordeal that came up of nowhere was shifting the entire setup to the new hdd. It made me remember of the osm vmware image that Milo van der Linden talked about in the early stages of the project. There was some real hike in the rendering speed compared to my 5-yrs old celeron 1.7 GHz mobo.

But now, my 6th semester exams are scheduled from 3rd-13th of June and i’m really getting low working hours now I will also put up a wikipage regarding i18n sooner than later. Loads of work still left…

Among other news, my activity oriented tutorials are up on the edu.kde.org …lots of thanks to Anne Marie (annma) for putting them up. I just love the logo guy pointing to the tutorials. Hopefully, i’ll try to put more of them. And more recently Debayan from NIT-DGP has put up the 3rd ntp pool server for India (with my WBUT starting the spur as always). I can now three ntp servers in http://www.pool.ntp.org/zone/in.

Update: Also as a matter of fact, globalize-rails now supports much more languages compared to ruby-gettext. And when combined with click-to-globalize plugin, i can see that we can even actually chip in translations from a WebUI….which is pretty cool

Yesterday, as an interesting twist of events my Google SoC proposal for OSM was accepted. Its’ about “Internationalization setup of osm web pages and map tiles“. Thanks to my mentor Mikel Maron and the entire OSM community. Its’ going to be a rocking summer

Internationalization of openstreetmap have diverse aspects hidden in it ranging from web pages localization, rendering localized map tiles including rendering of complex Indic scripts. And we will take up a phased approach and see what’s possible within the stipulated timeframe.

KGeography is a geography learning tool integrated with KDE-Edu. It helps a student to learn geography in a playful way. But it still doesn’t has all the maps as yet. How to add them? For adding a new map in KGeography, we need two things. One, the map (say in a file called mapname.png) and secondly, a kgm file (whose name should be mapname.kgm) to integrate the map with kgeography. Once we have them we can easily add them in its’ home directory (default: /usr/share/apps/kgeography). Now, what kgeography does is an image-processing. It distinguishes a division of the map by its’ RGB values. Then it identifies the division by Name, Capital, Flag etc parameters given in the kgm file for that division.

So, for adding a map we need to do following things:

1. First, create the map using GIMP, Kolourpaint or any other picture editors. Getting a political outline will be the best idea. I couldn’t get one atleast for India. So a good idea will be superimpose a blank outline map over a descriptive political map (in two layers to be precise). Assign separate colours for every division. Care should be taken that no two regions should have same colours. Ofcourse, handdrawn & scanned political outline maps may be an option and there is no real restriction on the scale of the maps.

2. Next, comes the integration part or the kgm file. So lemme show the basic structure of the .kgm file:

<map>

<mapFile>mapname.png</mapFile>
<name>MapName</name>
<division>

<name>DivisionName</name>
<flag>flagname.png</flag>
<capital>CapitalName</capital>
<color>

<red>255</red>
<green>255</green>
<blue>255</blue>

</color>

</division>
...............................
//Other divisions
...............................

</map>

A kgm file begins with <map> and ends with </map>. Inside of those tags there are:

• <mapFile> and </mapFile>: the name of the file (without any path) containing the map image, e.g. “india.png”.
• <name> and </name>: the name of map, e.g. “India”.
• A <division> and </division> for each division in the map.

Now, each division has these tags:

• <name> and </name>: the name of the division, e.g. “West Bengal”.
• <capital> and </capital>: the name of the capital of the division, e.g. “Kolkata”.
• <ignore> and </ignore>: can be set to yes, allowClickMode and no. If this tag is set to yes then the division will be ignored when asking for divisions in that map. It is then as if this division does not exist in KGeography quiz modes. If this tag is set to allowClickMode then KGeography will ask for the division in the Click Division in Map… mode but not in other quiz modes. Setting this tag to no means the division will appear in all modes (browse and quizzes). This tag is optional and when there is no <ignore> tag that means as it is set to no. For example “Egypt” has this tag set to yes in the “Asia” map which means that “Egypt” will not be part of any quiz in the “Asia” map. Also this tag is set to yes for frontiers or oceans.

<division>

<name>Not a Nation</name>
<ignore>yes</ignore>
<color>

.........................................

</color>

</division>

• <flag> and </flag>: the file (without any path) containing the flag of the division, e.g. “india.png” in asia map. This tag is optional. Not needed for the divisions whose <ignore> tag is set to yes.
• <color> and </color>: the color the division has in the map.

The color is defined using three tags:

• <red> and </red>: red component of the color. Valid values are between 0 and 255.
• <green> and </green>: green component of the color. Valid values are between 0 and 255.
• <blue> and </blue>: blue component of the color. Valid values are between 0 and 255.

All names should be in english. But when number of divisions are too much it will be better to automate the task of creating kgm file with a bash script (will post lil’bit later). I checked out the helper tool in kgeography handbook which wasn’t effective enough even on the existing maps.

Update: Now, this post may not be enough to dive into this work. So, I created a sort of presentation concentrating on how to add the map of India. It’s still unfinished. But you can have the first cut from here: [http://makghosh.googlepages.com/addmaps.pdf]. The best way to see it is as a presentation in kpdf.

Well, I divided the task in 3 days:

1. Day 1: Create the political map of India (done)
2. Day 2: Color the divisions of the map (done)
3. Day 3: Create the .kgm file (not completed)

Have fun!! And do post any comments or improvements

Lately I was trying to create a shutdown/reboot module for Fl_TeacherTool. I recently talked with its developer Robert Arkiletian and realised that “ltspinfo” can be accessed by non-root users even. The shutdown/reboot module is a cool feature for fl_teachertool but would create a problem if non-root users gain access and shutdown terminals at their will. But this problem has to solved. So, I am analyzing the code of ltsp-ltspinfod and may make changes in it as required to curb this problem.