← Home

Some common questions on getting started - Get on Rails with Globalize! Part 2 of 8

Let's talk Unicode

As you've enabled your application to virtually support even the strangest languages on earth you now should start using Unicode throughout your application so that non-latin characters will be stored and displayed correctly. If you don't, you'd probably get something like this:

Globalize.Unicode.Broken

... instead of this:

Globalize.Unicode.Ok

Globalize already includes jcode.rb and sets $KCODE to UTF8 (see vendor/plugins/globalize/init.rb) which basically means that Ruby will treat all input and output as Unicode encoded. So that's already been taken care of.

(Beware though that there are several String methods that won't work as expected with Unicode yet like length, reverse or slice. This is being worked on and it is neither Globalize's nor JCode's fault but due to Ruby's own lack of Unicode support which reportedly will be fixed in Ruby 1.9/2.0.

As this will take some time Rails will stand in with ActiveSupport::Multibyte - which will be included in the upcoming Rails 1.2 release (thanks, Josh!) and allow all the standard String methods to work in Unicode. Watch a screencast about ActiveSupport::Multibyte here.)

So, essentially, to get Unicode being used throughout your app you'll primarily need to make sure that it's being used by a) the database and b) the browser.

... to your database ...

To tell Rails that you want to use Unicode with your database add the line in config/database.yml:

 encoding: utf8    # for MySQL
 encoding: unicode # for PostgreSQL

For MySQL that's the eqivalent to: SET NAMES utf8. For PostgreSQL it does: SET CLIENT_ENCODING TO unicode. For SQLite you just need to compile Unicode support in.

Also, you additionally need to configure MySQL itself to use utf8 as character set. There are some ways to do this. According to the MySQL Manual (see sections 10.3.1 through 10.3.4) you can set the character set at server, database, table and even column level. More specific settings overwrite a more general setting. E.g. you could do:

CREATE TABLE articles (
  ...
) CHARSET=utf8;

... which would work. As you're going to use migrations though (you do, don't you?) you need to ensure that MySQL uses utf8 as the default character set for new tables - and you can achieve that by setting the character set at the database or even server level. For the database that's as simple as:

[CREATE|ALTER] DATABASE db_name
	CHARACTER SET utf8

Also, you'll probably want to pay attention to your collation. The database server's collation defines the way your data is sorted - which may cover all kinds of wired conventions. (For example, see this section in Wikipedia for some wired variations of sorting conventions regarding German Umlauts). You can set the collation you want to use in one go with the character set. E.g.:

[CREATE|ALTER] DATABASE db_name
	CHARACTER SET utf8
	COLLATION utf8_spanish_ci

... and to your clients

Next make sure that your application's clients (i.e. the user's browser) will decode your output (like HTML ...) correctly. The safest (and standard-compliant) way is to send an appropriate header:

class ApplicationController < ActionController::Base
  before_filter :set_charset
  def set_charset
      content_type = headers["Content-Type"] || "text/html"
      if /^text\//.match(content_type)
        headers["Content-Type"] = "#{content_type}; charset=utf-8"
      end
  end
end

This will add the Content-Type header to our http response and signalize that we'll be sending Unicode to the client. This also ensures (in modern browsers, that is) that the browser will return stuff encoded as Unicode (e.g. data sent as forms).

Also, to be nice to Safari add a Content-Type meta tag to your layout templates (such as default.rhtml):

<meta http-equiv="Content-Type" content="text/html; charset=utf-8">

How to select and keep the current user's locale

Like you've seen in Part 1 you can set the current locale by using Locale#set (e.g. Locale.set("de-DE")). But how do you know which one to set? There's no one-fits-all answer, it depends. But here are some options you'll probably want to think about.

To determine the users locale you could:

  1. use a locale that's been suggested by the browser (using the accept-language header)
  2. use a locale the users ip address seems to belong to
  3. offer (or force) the user to choose a locale manually
  4. use a default locale otherwise.

Once the users current locale is set (be it that you've decided to present a default language or that the user already has chosen the preferred language) you need to persist it throughout her requests somehow. Basically your options are:

  • Keep the locale in the URL as a ...
    • parameter (like in http://site.com/home?lang=pl)
    • part of the path (like in http://site.com/pl/home)
    • subdomain (like in http://pl.site.com/home) or domain (like in http://site.pl/home)
  • Store it in the session or a cookie.

Now, to set your users locale you'd most likely do something like this in your ApplicationController:

before_filter :set_locale
def set_locale
  # somehow determine and set the locale
  true
end

For example the following has been suggested:

This code sets the Globalize locale based on the http accept-language header field. Other options are also considered, in order or precedence:

  • Explicetely locale set on URL
  • Previous language selection (stored on user session)
  • Accept-Language field contents
  • Default locale
before_filter :set_locale

def set_locale
  default_locale = 'en-US'
  request_language = request.env['HTTP_ACCEPT_LANGUAGE']
  request_language = request_language.nil? ? nil :
    request_language[/[^,;]+/]

  @locale = params[:locale] || session[:locale] ||
            request_language || default_locale
  session[:locale] = @locale
  begin
    Locale.set @locale
  rescue
    Locale.set default_locale
  end
end

Also, you might want to consider to improve this solution by checking the locale against an array of locales that are available in your application, like so:

locale = LOCALES.join(' ').match params[:locale] || default_locale

Beware though that using the session for something like this might yield unexpected behaviour (even if it might seem invitingly simple). In usability tests we've often seen users getting confused by this type of solution when they are switching back and forth between multiple browser windows. When a user changes the locale in one window it won't change in the other windows of course - until he does the next click (or refresh). This type of behaviour might (probably depending on the nature of your application) lead to confusion and disorientation of some degree.

So, all in all you might be better off to avoid usability gotchas like these altogether, resort to a more RESTful approach and encode your locale directly in your URLs. When you're going this route you also might find it useful to redirect the user when no locale parameter is provided. E.g., Szymek suggests in his Globalize example application:

def set_locale
  if !params[:locale].nil? && LOCALES.keys.include?(params[:locale])
    Locale.set LOCALES[params[:locale]]
  else
    redirect_to params.merge('locale' => Locale.base_language.code)
  end
end

How to localize your templates

You're able to translate strings in templates using .t. But how can you localize entire templates?

Why you'd want to do that? A common example for this requirement is the need for right-handed layouts for languages such as Arabic and Hebrew. It might be sufficient to check Locale.active.language.direction() to conditionally load different style sheets using a before_filter in your ApplicationController in some cases. In other cases this just won't cut it though.

Globalize therefor looks for localized templates for the current language using the following naming scheme:

[templatename].[lang].rhtml

If it does not find a template named like this, the default [templatename].rhtml file will be used.

Of course this approach is most oftenly less DRY than just using one template for the markup and control the layout of the page through CSS - but that's not always possible.

How to translate Rails ActiveRecord messages

This is probably one of the most frequent asked questions.

Joshua Harvey writes: "[...] Globalize was designed to handle this. Globalize automatically tries to translate ActiveRecord validation messages, so if you trigger a validation error, that message should show up in the globalize_translations table and be available for translation."

Sounds cool. Let's have a look at this.

First let's create a model, e.g.:

class Page < ActiveRecord::Base
  translates :title, :text
  validates_presence_of :title
end

Now switch over to your script/console terminal and force an error message:

>> Locale.set "de-DE" # without an active Locale nothing interesting would happen
>> p = Page.new
>> p.save!
ActiveRecord::RecordInvalid: Validation failed: Title can't be blank

Great, let's check the database:

>> Translation.find :first, :order => "id DESC"
=> #<Globalize::ViewTranslation:0x2657ba0 @attributes={...,
   "tr_key"=>"Title can't be blank", ..., "language_id"=>"1556", ...}>

Bingo. Globalize has inserted an empty recordset for the key "Title can't be blank" into the globalize_translations table. The language with the id 1556 is German.

Joshua writes: "Because Globalize's string substitution isn't quite as sophisticated as we want it to be, it can't translate field names separately from the validation message. This means you have to translate all combinations of field name/validation type. In many apps this isn't that big a problem, but if it is in your case, let us know."

Of course there's more ...

In Part 3 of this series we'll have a look at Globalize's more advanced features like:

  • Abstracting ViewTranslations (sprintf-like usage)
  • Singular and (multiple) plural ViewTranslations
  • Globalize's Currency class
  • Piggyback translated ActiveRecord associations