The Future of I18n in Ruby on Rails - RailsConf Europe 2008

Welcome everybody!

I'm very happy to talk about "The future of Internationalization in Ruby on Rails" today.

So, who’s me?

This never occurred to me before but in a recent RailsEnvy podcast I’ve learned that my name could be pronounced like this ...

If you want to google me ...

... you probably have better luck using this.

I'm currently living in Berlin and I love it :)

I started programming in 1984

I've been in business as a developer, mostly as a web developer for some years and switched to working with Ruby and then Ruby on Rails a couple of years ago.

Some of you might know me because I've led the Rails I18n group for the last couple of months and did the final touches on the implementation of the new API in Rails recently.

Some people know my blog because of some tutorials I've written about Globalize.

And I’m currently working on a CMS application platform called adva_cms.

Of course, as a German developer I've always been interested in support for Internationalization so I'm really happy that I’ve able to contribute something in this area to Ruby on Rails.

If there are any questions please try to make a mental note and keep it for the end. I've set some time aside for Q&A at the end of the talk.

Ok, by the end of this talk I hope you’ll have a clear picture of what the Rails Internationalization project is, what’s included in the new Rails Internationalization API and what’s not and how you can use it.

I will give a very brief demo. Of course this is a huge topic so I’ll point you to where you can find more resources.

Then we have a short and sweet Interview video prepared with Joshua Harvey the original author of Globalize

And finally I’ll give you some hints about what we’re going to do in the future and how you can contribute to that.

So, to better understand the future of Internationalization in Rails I want to contrast that a bit with the history that we all suffered through. As you know the history of Internationalization in Rails is really, really dark.

These were times of chaos.

They were unclear, unpredictable, unstable and highly adventurous.

You'll see the reason for that in a minute. Let me first let you show how many different solutions we've already seen and how wide their variety is.

We have seen things like ...

Ruby Gettext - the dinosaur and grandfather of Internationalization

David’s original Localization plugin as well as ...

Thomas Fuchs’ Localization plugin - both very simple

GLoc - one of the first bigger implementations

Globalize - the only solution with solid Model translations

Globalite - a very slick solution by Matt Aimonetti

Simple Localization - also a very interesting solution by Stephan Soller

Both have contributed a lot to the Rails I18n project by the way.

Gibberish - maybe the most recent full solution, came along as the cool kid on the block.

... and many more.

There are solutions for special needs like ...

Localizing Rails to like Swedish, Brazilian, Korean and so on.

There are plugins that add functionality to other plugins. For example there is Jibberish that adds a Javascript Layer on top of Gibberish. And things that just adds separate, useful tools.

Stuff like that ... of course there’s much more. I just wanted to give you an impression of these things.

So when you take into account how complex our languages are and how many funny problems there are to solve and we have lots of lonely programmers working on this and everybody focusses on the features he actually needs himself ...

What do you think will you get?

What you get is a huge mess.

We have lots of isolated, incompatible solutions.

Things like support and documentation just don’t get better with a lonely developer solving their own needs and then - of course - often walking away and focussing on something completely different.

Then, all of our solutions basically were huge Monkeypatches to Rails with the usual problems resulting from that.

And all of us re-implemented the wheel in one way or the other. So, in a way, we wasted lots of resources.

But - on the other hand - this situation was actually necessary for us to experiment with solutions and learn how Rails Internationalization could be done.

Because of this situation last year in September a group of developers from several Rails Internationalization plugins gathered for a chat session. We agreed that we wanted to implemented a Rails core patch which should solve the problems we had with this stuff.

We invited developers from all existing solutions that we knew and started meeting regulary, collected requirements, discussed our ideas ...

Actually there was really a lot of discussion and initially it was hard to agree on anything.

You know that sort of situation. Everybody brings their own favorite little features to the table, everybody has different beliefs of what's good and what's necessary. So this discussion went on endlessly and in the beginning of 2008 our work slowed down. Apparently we really needed a creative break and some time to rethink everything.

Then, in May 2008 we got back to the drawing board with a fresh approach ...

... and now we were able to agree on a very slim and lightweight implementation that covered all of our requirements

It works as a Ruby gem which means it is not only suited for use in Rails but also other frameworks.

We wanted only minimal changes in Rails, we wanted it to add as less load to Rails as possible and of course we still wanted it to implement a common API where everybody could build on.

So, at this point Jeremy Kemper didn’t really know about our work but in a RailsConf Portland RailsEnvy interview he predicted for Rails 2.2 that ...

... Internationalization will be solved.

When we asked Jeremy about this he said it was a joke

So we think Jeremy should definitely crack more jokes about things like this.

Because in the end this joke actually got us going to finally propose our work to the core team.

In hindsight there were a few key decisions. Before we managed to agree on these we had not been able to get anything really done. These key decisions were:

We agreed to make the actual implementation of all the logic exchangeable, so you can always plug in something more powerful or more flexible if you need to.

We radically sticked to the principle of “doing the simplest thing that ever could work” for Rails.

And maybe most importantly we just stopped aiming for a solution that did anything else than the bare minimum that’s needed for Rails.

So, what we now ship with Rails is a common API for Internationalization which is basically just a bunch of definitions how things are supposed to work. So this is just a Ruby module with some public methods on that.

These methods delegate to a backend that actually implements the API. The backend that is shipped with Rails is intentionally named the Simple backend because the only requirement for the Implementation in the Simple backend is that it works for American English because that’s what we need for Rails to keep it working like it worked before.

That means: the most important parts that previously were hardcoded right into Rails are now abstracted out of the code and moved to a simplistic backend, accessible through just a few public methods.

So this is true

While this ... might work. Maybe you can localize Rails to your favorite language if it is very similar to English. There are quite some resources out there that show how to do this. For everything else we still need to add plugins or other libraries.

For Rails itself we settled with shipping the translations as YAML files and with stock Rails you can also use Ruby files, so you can have lambdas in your translation data.

I’ll run through the main features that this API provides.

As I said the API is provided as a Ruby module with two main methods on it.

These are:

I18n.translate which you can use to look up translations from the backend and for convenience it has an alias #t and you need to pass it at least one parameter which is the key.

So this would look up the translation that’s stored for the key :rails in the current locale, for example, by default American English.

The other method that you’ll use regularly is localize.

This one basically allows you to format Date and Time objects for a certain locale.

Again, for convenience there’s an alias ... of course that’s #l and you pass it the object that you want to localize

So the #translate gives you very powerful means to look up translations.

You can look up your translations using a Symbol as a key and this is what we’d generally encourage you to do. But you can also use a String.

Also, you have scopes, so you can namespace your translations and structure them in a reasonable way.

For example in the ActiveRecord translations YAML file there’s a scope activerecord and then there’s a scope errors, a scope messages and finally there’s finally the translation for the key :invalid.

So scopes are basically groups or namespaces of translations. You can think of them as nested Hashes

Consequently there’s a way to look up whole namespaces - or groups - of translations. For example this will look up all error messages in ActiveRecord and return a Hash with all those messages.

Then there’s bulk lookup. That just means that you can pass an Array of keys you want translated and it will return you a corresponding array of translations.

And finally, of course, there’s defaults so you can specify what happens when a translation can not be found.

For example when you give it a String as a default and the translation can not be found it just returns the String.

But when you give it a Symbol it will translate this Symbol and give you that translation as a default.

Finally you can specify an array of Symbols and/or Strings and it will just check everything in that order and return the first value that’s not nil.

Interpolation is another feature.

That means you want to abstract your translations in certain situations. For example you have a translation for the key :message and there’s variable :name in that. Then you can pass the value of that variable to #translate and it will parse the value into the translation.

This looks trivial but it’s actually a very important feature because concatenating translations is kind of an anti-pattern in many situations.

Also, the API provides means to define Singular and Plural translations of course.

For example you could have a Singular and a Plural translations like this. Then you can pass a count option to #translate. So in this case it would pick the Singular translation because count is 1. And when you pass it 2 it would pick the Plural translation.

So that’s it already - all very basic stuff of course. No worries, you can look up everything online :-)

Like I said, the I18n API ships with a Simple backend ...

... which implements all of these features for American English only. So for example you can’t use it to do pluralization for Czech because they have different rules for pluralization.

As I just showed you your English translations include something like this.

But in Czech you have two plural forms - one of them is called Paucal. They use the singular just like one does in English. Then they use the Paucal, the second form, for everything between 2 and 4. And the actual plural for everything else.

So the Simple backend can not do that but we can easily swap it and the Internationalization API actually encourages you to do so.

You would just create a class for your custom backend and maybe extend the Simple Backend, implement your custom logic and tell the I18n module to use it.

That’s it.

Basically this is what future plugins will do a lot. For example there definitely will be a backend that stores translations to the database and I’m sure there’s one soon to read and store them from and to Gettext files.

So let’s have a really brief look how you can use this in your application.

So, if you’re actually working on an application today, what can use use?

Of course this really depends on your requirements. So let’s have a look at some scenarios ...

Let’s say you just need a few static translations and maybe some date and number formats. Then you can easily reuse the Simple backend that Rails ships with. By now there are a couple of demo applications on GitHub that show how to do that.

Let’s say you have a language that requires different pluralization rules. Then there’s an experimental Pluralizing backend for that where you can hook in pluralization logic as lambdas.

Let’s say you need to translate your application to Russian, or Czech or another language that has some more complex date formats.

For that there’s an experimental Chain backend where you could hook in a custom backend tailored for just these languages’ date formats.

Also there’s already a full plugin solution by Yaroslav Markin from Russia that implements things like these and I think it’s easy to adapt it for other languages.

If you need model translations there’ll be Globalize2. Globalize2 will look completely different to Globalize. It will be pretty lightweight and unobtrusive. Joshua will tell you more about that in the interview.

Also there’s an alternative approach, that’s linked on rails-i18n.org

So, if we still have to rely on external code, like plugins, to localize our applications.

What do we win?

The short answer to that clearly is: we win a lot!

First of all, we don’t need no monkeypatching any more

Well, of course we still will monkeypatch Rails to experiment with new features.

But we definitely won't need to do probably 99% of what we’ve done before like translating ActiveRecord error messages, translating all the helpers that Rails provides or anything like that.

Also, because we now can build on a common API we expect things to become much better exchangeable.

We will aggregate common locale data over time. So we will build up a Rails specific pool of Localization knowledge.

Also we hope that plugins will be much more focussed on implementing certain atomic features.

This is actually something we encourage everybody to do. You know, so far, you could not possibly combine three, four, five different plugins and expect anything to work. We hope we’re now able to do a better job here in future.

So, of course, with all of this stuff released our next development cycle has started ... right now.

Of course we’re all going to implement our stuff next. In fact by now there are already quite some useful things out there. During this time we will try to stay in touch with each other and then, again, collect everything, review it, again, discuss it and finally aim for another core patch.

The reason why we want to first collect everything and look at it as a whole and then move it to Rails as a rather complete package is that many of these things make most sense in context with each other.

We hope this will then result in a landscape of Rails Internationalization solutions where we have a distributed toolbox of focussed plugins rather than those huge solutions trying to solve everything under the sun at once that we had before.

We’ll have more compatible solutions and maybe the most exciting thing is that we will get to have Rails specific, common locale data.

So, let’s see what Joshua Harvey has to say about Globalize 2.

By the way the guitar shop that you can see in the video is actually the shop rashgash and the website at rashgash.com is the reason why Joshua initially wrote Globalize as he mentions in the interview.

So this is a brief overview of features that Globalize 2 will provide.

Of course there’s model translations - kind of Globalize’s USP in the past. For View translations we’ll basically use the Rails I18n API. There will be some useful tools like standards compliance tool for locales which we’ll need for locale fallbacks and stuff like that. A locale load_path tool that helps loading translation data more easily.

Actually the implementation of Globalize 2 is sponsored by the company BESTGroup Consulting & Software from Berlin. So, thank you, BESTGroup for making this possible.

Also, Metaversum, a company also from Berlin will start sponsoring us right after this conference. So, thank you, Metaversum in advance.

And actually we are looking for more sponsors not only for the Globalize project but also for the Rails I18n project because we really want to put this project on a really solid foundation.

For example we want to have somebody working on stuff like documentation on a regular basis and we want to provide better support for newbies on the mailing list and other stuff that is all usually much easier to do when you have a small budget to spend.

So if you’re interested in sponsoring a really cutting edge and high-profile Ruby on Rails Open Source project that already gets tons of attention and definitely will get even more attention in the next couple of months ... just catch me afterwards and we can talk about that.

So, how can everybody in this room get involved?

Of course everybody can just start playing with all of this, register to our mailinglist and give feedback. We need tons of that.

You can share your translations. We already have a repository on GitHub for that, so if you’ve done translations for Rails please submit them.

You can build plugins focussed on certain features that are still missing. And please don’t forget to pitch them on the mailinglist and our wiki.

You can help improving the Rails integration. You know Rails helpers are there and they are working. But that doesn’t mean they’re already perfect regarding Internationalization. So you can help to improve that.

And of course, you can sponsor our work so we can invest some money in things that are hard to do otherwise.

So, basically this was my talk. We’ve collected pretty much everything I mentioned in this talk as well as lots of other resources on http://rails-i18n.org. So, look it up.

Thanks to everybody who contributed!

Ok, so what are all those questions you want to ask?