← Home

Experimental Ruby I18n extensions: Pluralization, Fallbacks, Gettext, Cache and Chained backend

Backend::Simple < Backend::Base

In Ruby what is the most obvious, elegant and maintainable pattern to extend an existing class' or object's functionality? No, the answer to that is definitely *not* in using alias_method_chain. It's simply including a module to that class. You probably knew that already ;) We've done this with the I18n Simple backend before but one needs to extend the Simple backend first in order to then inject additional modules to the inheritance chain so that these modules' methods would be able to call super and find the original implementation. To make this a bit easier I've moved the original Simple backend implementation to a new Base backend class and simply extend the (otherwise empty) Simple backend class from it (see here and here). This way you now do not need to extend the Simple backend class yourself but you can directly include your modules into it: 
module I18n::Backend::Transformers
  def translate(*args)
    transform(super)
  end

  def transform(entry)
    # your transformer's logic
  end
end

I18n::Backend::Simple.send(:include, I18n::Backend::Transformers)
I have no clue what your Transformers module could do exactly but that's the point about extensible libraries, isn't it? In any case this is simply the pattern that the new, experimental Pluralization, Fallbacks, Gettext and Cache modules use that I wanted to talk about :)

Pluralization

Out-of-the-box pluralization for locales other than :en has been recurring requests for the I18n gem. Even though it was easy to extend the Simple backend to plug in custom pluralization logic and there are working backends doing that (e.g. in Globalize2) there does not seem to be a point in still rejecting a basic feature like this from being included to I18n. I've thus added a Pluralization module that was largely inspired by Yaroslav's work. It can be included to the Simple backend (or other compatible backend implementations) and will do the following things: * overwrite the existing pluralize method * try to find a pluralizer shipped with your translation data and if so use it * call super otherwise and use the default behaviour One can ship pluralizers (i.e. lambdas that implement locale specific pluralization algorithms) as part of any Ruby translation file anywhere in the I18n.load_path. The implementation expects to find them with the key :pluralize in a (newly invented) translation metadata namespace :i18n. E.g. you could store an Farsi (Persian) pluralizer like this: 
# in locales/fa.rb
{ :fa  => { :i18n => { :pluralize => lambda { |n| :other } } } }
And include the Pluralization module like this: 
require "i18n/backend/pluralization"
I18n::Backend::Simple.send(:include, I18n::Backend::Pluralization)
We still have to figure out how to actually ship a bunch of default pluralizers best, but you can find a complete list of CLDR's language plural rules compiled to Ruby here (part of ours test suite).

Locale Fallbacks

Another feature that was requested quite often, too, is Locale fallbacks. Simple backend just returns a "translation missing" error message or raises an exception if you tell it so. It won't check any other locales if it can't find a translation for the current or given locale though. There were proposals for a minimal fallback functionality that just checks the default locale's translations if a translation is not available for the current locale. Globalize2 on the other hand ships with a quite powerful Locale fallbacks implementation that also enforces RFC 4646/47 standard compliant locale (language) tags. I've discussed this with Joshua and we've decided to extract a simplified version from Globalize2's fallbacks that makes the RFC 4646 standard compliance an optinal feature but still allows enough flexibility to define arbitrary fallback rules if you need them. If you don't define anything it will just use the default locale as a single fallback locale. Again enabling Locale fallbacks is just a matter of including the module to any compatible backend: 
require "i18n/backend/fallbacks"
I18n::Backend::Simple.send(:include, I18n::Backend::Fallbacks)
This overwrites the Base backend's translate method so that it will try each locale given by I18n.fallbacks for the given locale. E.g. for the locale :"de-DE" it will try the locales :"de-DE", :de and :en until it finds a result with the given options. If it does not find any result for any of the locales it will then raise a MissingTranslationData exception as usual. The :default option takes precedence over fallback locales, i.e. it will first evaluate a given default option before falling back to another locale. You can add custom fallback rules to the I18n.fallbacks instance like this: 
# use Spanish translations if Catalan translations are missing:
I18n.fallbacks.map(:ca => :"es-ES")
I18n.fallbacks[:ca] # => [:ca, :"es-ES", :es, :en]
If you do not add any custom fallback rules it will just use the default locale and the default locales fallbacks: 
# using :"en-US" as a default locale:
I18n.default_locale = :"en-US"
I18n.fallbacks[:ca] # => [:ca, :"en-US", :en]
If you want RFC 4646 standard compliance to be enforced for your locales you can use the Rfc4646 Tag class: 
I18n::Locale::Tag.implementation = I18n::Locale::Tag::Rfc4646
This will make a locale "de-Latn-DE-1996-a-ext-x-phonebk-i-klingon" fall back to the following locales in this order: 
de-Latn-DE-1996-a-ext-x-phonebk-i-klingon
de-Latn-DE-1996-a-ext-x-phonebk
de-Latn-DE-1996-a-ext
de-Latn-DE-1996
de-Latn-DE
de-Latn
de
Most of the time you probably won't need anything like this. Thus we've used the (much cheaper) I18n::Locale::Tag::Simple class as the default implementation. It simply splits locales at dashes and thus can do fallbacks like this: 
de-Latn-DE-1996
de-Latn-DE
de-Latn
de
Should be good enough in most cases, right :)

Gettext

The difference between the Gettext support and all the other extensions discussed here is that I haven't had a real use for it myself - so far, so please consider this stuff highly experimental. It shares the fact that people requested it in one way or the other though, so I'd appreciate feedback about it. Gettext support comes with three parts, only two of them being relevant to the user: * classical Gettext-style accessor helpers * a Gettext PO file compatible backend storage (currently read-only) * a few internal helpers To include the accessor helpers to your application you can simply include the module whereever you need them (e.g. in your views): 
require "i18n/helpers/gettext"
include I18n::Helpers::Gettext
The backend extension is, again, a matter of including the module to a compatible backend (e.g. Simple): 
require "i18n/backend/gettext"
I18n::Backend::Simple.send(:include, I18n::Backend::Gettext)
Now you should be able to include your Gettext translation (\*.po) files to the I18n.load_path so they're loaded to the backend and you can use them as usual: 
I18n.load_path += Dir["path/to/locales/*.po"]
Please note that following the Gettext convention this implementation expects that your translation files are named by their locales. E.g. the file en.po would contain the translations for the English locale.

Translate Cache

Right from the beginning people benchmarked the I18n gem implementation and compared their numbers against native Hash lookups or other, less rich implementations of similar APIs. Also, thedarkone has experimented with a Fast backend that implements some significant optimizations. For the I18n gem itself we've refrained from overly extensive "early optimizations" and only applied some tweeks that made the implementation cheaper in obvious ways. This rather conservative approach actually paid out: the clean and readable implementation made the Simple backend refactorings and extensions (like those discussed here) almost trivial. On the other hand, even though calls to I18n don't actually add that much load to an application sparing resources obviously is an important concern. The probably both most effective and least intrusive way of doing that is simply caching all calls to the backend. Again you can include the Cache layer by simply including the module: 
require "i18n/backend/cache"
I18n::Backend::Simple.send(:include, I18n::Backend::Cache)
As we do not provide any particular cache implementation though you also have to set your cache to the I18n backend. The cache layer assumes that you use a cache implementation compatible to ActiveSupport::Cache. If you're using this in the context of Rails this is a matter of one line: 
I18n.cache_store = ActiveSupport::Cache.lookup_store(:memory_store)
Obviously this pluggable approach again allows you to pick whatever cache is most appropriate for your setup. For example ActiveSupport out of the box ships with compatible implementations for plain memory, drb demon and compressed, synchronized and plain memcached storage - wow. These options should be sufficient for 99% of all Rails apps. Whatever cache implementation you use the I18n backend cache layer will simply cache results from calls to translate (and will do the right thing with MissingTranslationData exceptions raised in your backend). For that it relies on the assumption that calls to the backend are idempotent: "A unary operation is called idempotent if, whenever it is applied twice to any value, it gives the same result as if it were applied once.". I18n's library design does not garantuee that by itself but in all practical cases will behave this way unless your doing really weird things with it. Basically, make sure you only pass objects to the translate method that respond to the hash method correctly. If you use custom lambda translation data make sure they always return the same values when passed the same arguments.

Chained backend

The Chain backend is another feature ported from Globalize2. It can be used to chain multiple backends together and will check each backend for a given translation key. This is useful when you want to use standard translations with a Simple backend but store custom application translations in another backends. E.g. you might want to use the Simple backend for managing Rails' internal translations (like ActiveRecord error messages) but use a database backend for your application's translations. To use the Chain backend you can instantiate it and set it to the I18n module. You can then add chained backends through the initializer or backends accessor: 
# preserves an existing Simple backend set to I18n.backend
I18n.backend = I18n::Backend::Chain.new(I18n::Backend::ActiveRecord.new, I18n.backend)
The implementation assumes that all backends added to the Chain implement a lookup method with the same API as Simple backend does.

Cool, what's next?

If you're interested in any of these features, please try these things out and provide some feedback on the rails-i18n mailinglist. They might make it into the next I18n gem release or not depending on the amount of "real world feedback" we've gotten until then. Also, by now there's ton of good stuff that uses or extends I18n to do useful things with it on Github. I've collected a few things that I found particular suited for being evaluated, maybe merged and potentially included to I18n here: Interesting I18n repositories. I've also done some work on my i18n-tools repository recently, so you hopefully you can expect some news from that front, too.

Shameless plug

In case you find this stuff useful I'm always happy to receive another recommendation on working-with-rails :)