Vesuvius - Google Summer of Code 2012 Ideas

Drafting the Proposal: Full Support for Multiple Languages

Background: GSoC 2011 Precursor 'Pootle' Project

Related Google documents (some which may be found through the preceding links) include:

Note that the 2011 work included some direct interaction with the Pootle system, but the 2012 work is much more focused on the Sahana PHP codebase.

Recent Correspondence on the List About This 2012 Idea with Ramindu

This email thread, edited for concision and presented as a dialog, began on discuss@lists.sahanafoundation.org

3/19/2012

Ramindu: I managed to work out some problems I was having with configuring .htaccess for Vesuvius as well as integrating the translation system in to the new configuration and new code of Vesuvius. A few areas on which I could work on as I see them are:

  • The new Vesuvius code seems to be breaking the Google Translations. This needs to be looked in to and fixed. A better migration towards using POST requests rather than GET requests to retrieve Google Translations needs to be formulated as well.
  • At present, users are required to run a build_mo bash script in order to generate *.mo (Machine Object files read by the PHP gettext library for string substitutions) files. I would suggest building the functionality in to the backend of Vesuvius that would simply carry out this task without leaving extra work for the user. As I see it now, this would mean executing a shell script from within PHP. As this is not allowed on most servers, it should be a nice challenge to develop a workaround.

The above are what I see as the most pressing matters related to this project in addition to what is listed on the ideas page….

3/22

Ramindu: I'm thinking of starting work on my proposal for the project, and I will outlining the following in my plan:

  • Bringing the translation framework up-to-date with the most recent changes to Vesuvius
  • Making the translation framework version-independent, so that it will not break again when changes are made to the structure of Vesuvius.
  • Getting a consensus on the contexts that need to be added to strings, and doing major work towards adding those contexts to the strings of Vesuvius.
  • Building a module/functionality in to Vesuvius that will allow users to immediately apply a *.po file (via conversion to an *.mo file) to Vesuvius.
  • Better error and exception handling for translations.

Ideas and comments would be highly appreciated.

3/23

Glenn: That all sounds reasonable, and the right amount of work for a summer effort.

As you may recall, I’m also interested in a beyond-Pootle-grade translation system applied to specific resource pages/specific languages. (Greg has a hard time appreciating this, because he has more faith in Pootle results than I.) A sophisticated version of that is probably beyond the scope of this summer. But some way to “pin” pages and say “for certain languages, don’t overwrite this page with a Pootle translation, use this cleaned-up canned translation instead.” might be do-able.

3/25

Discussion is moved to agasti@lists.sahanafoundation.org

Ramindu: While this sounds like a good idea, I have a feeling that it might cause a drop in efficiency as the current gettext library makes use of Machine Object (binary) files to read and apply translations. Any other system we use to apply translations might slow it down.

However, during my work with Joomla, I saw that their translation system depends entirely on language.ini files placed alongside the various extensions of Joomla. These simply contained plaintext replacements for various base strings (e.g. USER_SUBSCRIBE would be replaced by 'Subscribe' in English). This method hasn't caused a noticeable drop in load times, so I guess it's feasible.

So if you're thinking of an internal translation mechanism that doesn't depend on the gettext library or Pootle, we could look in to that possibility.

Glenn: What I have in mind is not substitution at the phrase level, as Pootle or Joomla does, but at the entire-page level. This would only apply to certain pages that are primarily static explanatory text. Pootle is fine elsewhere. Probably no major efficiency issue in page substitution.

Ramindu: I see, so you're suggesting that we have full-page replacements (e.g. locale is set to 'French', it will load the full french page including code and hardcoded strings). Is that right?

Would that have anything to do with the current Resource Page structure, where the strings are stored in the database?

Glenn: Yup, it's mainly Resource Pages that would be candidates for this treatment. But there would have to be fallback to the Pootle approach for languages not covered by full-page treatment. You can see there are tricky versioning issues, but these could be handled manually in the near-term.

3/26

Ramindu: In the current mechanism, the Resource Pages are totally replaced by Google Translated strings, based on the availability of that particular language in Google Translate.

Since Resource Page text is stored in the database (to make it editable for Vesuvius admins), would it be practical to hardcode them as strings within the code?

Glenn: That would not be my preferred way to go. I'd like to define a new “editor” role, separate from admin, who would NOT be a programmer, but could be a context provider and/or translator. So having content separate from code (in either database or text file) would facilitate that.

Ramindu: Interesting. This would mean a separation of the contexts (at least in the case of these pre-defined Resource Pages) from the source code. Maybe total separation of contexts is in order? (this would make the context building process much quicker and more efficient).

If this project is accepted, looks like it will rely heavily on research, as it did last time.

We venture once more in to the territory of the unknown! ;)

Glenn: If by total separation you mean to include the non-resource pages, with their many controls, I think that would be too much pain for too little gain. I'm fine with Pootle alone for those. Also, I think what we are discussing is an “if time allows towards summer's end” aspect… much more important to get Pootle working across all modules. Maybe just coming up with a Resource Pages implementation plan is enough for this summer, rather than the actual implementation.

Ramindu: Ok, so what ill describe in the proposal is 'restructuring of Resource Pages to enable replacement by pre-translated, location-specific pages.' Does that sound like what you've been describing?

Glenn: Ultimately, I’d like resource pages to be event-specific too, so then the replace would be by pre-translated pages specific to a particular language and event. However, I don’t know that event specificity for Resource Pages will be achieved by this summer, so “language-specific” replacement may be the reasonable target.

3/27

Ramindu: So what we would like to do is load pre-translated resource pages based on locale, which would again be editable by users, instead of using Google Translate as is being done at the moment. I think that would be achievable to a certain degree within the summer.

Glenn: Yes, where “locale” is the chosen by the Vesuvius visitor from a pulldown (currently somewhat suppressed), and “editable by users” is true for admins/editors, but not general users.


QR Code
QR Code agasti:vesuvius:gsoc2012:multiple-languages (generated for current page)