Difference between revisions of "Coding for translation using weblate"

From Gramps
Jump to: navigation, search
m (fix redirect)
 
(7 intermediate revisions by 2 users not shown)
Line 3: Line 3:
 
= Introduction =
 
= Introduction =
  
Gramps has always been internationalized (see: https://gramps-project.org/introduction-WP/2006/04/looking-back-over-5-years/ ) Therefore, all strings meant for the user should always be flagged for translation.
+
Gramps has always been internationalized (see: https://gramps-project.org/blog/2006/04/looking-back-over-5-years/ ) Therefore, all strings meant for the user should always be flagged for translation.
  
 
In order to be considered for inclusion in the official Gramps release, any piece of code must support internationalization. What this means is that the Python module must support [[Translating Gramps|translations]] into different languages. Gramps provides support to make this as easy as possible for the developer.
 
In order to be considered for inclusion in the official Gramps release, any piece of code must support internationalization. What this means is that the Python module must support [[Translating Gramps|translations]] into different languages. Gramps provides support to make this as easy as possible for the developer.
 
= Adding a new language =
 
 
To add a new language, navigate to the "gramps" component in Weblate and click on the '''Tools -> Start new translation''' menu option.  Then select a language code.  You need to be a Weblate administrator to do this.
 
 
Then the language code must be added to the [https://github.com/gramps-project/gramps/blob/master/po/LINGUAS LINGUAS] file.
 
 
An entry must also be made in the <code>_LOCALE_NAMES</code> dictionary in the [https://github.com/gramps-project/gramps/blob/master/gramps/gen/utils/grampslocale.py#L70 grampslocale.py] file.
 
  
 
= How to allow translations =
 
= How to allow translations =
Line 37: Line 29:
  
 
== Plurals ==
 
== Plurals ==
 +
 +
Plurals are handled differently in various languages. Whilst English or German have a singular and a plural form, other languages like Turkish don't distinguish between plural or singular and there are languages which use different plurals for different numbers, <abbr title="exempli gratia - Latin phrase meaning 'for example'">e.g.</abbr>, Polish.
 +
 +
Gramps provides a [[Translating_Gramps#Plural_forms|plural forms]] support, useful for locales with multiples plurals according to a number (''often slavic based languages'') or for Asian family languages (''singular = plural'').
 +
 +
Note, some locales need singular form with [http://en.wikipedia.org/wiki/Plural#Zero zero] and plural form might be also used in this case.
  
 
In some strings, it's necessary to specify different translations depending upon the number of an argument. For example,
 
In some strings, it's necessary to specify different translations depending upon the number of an argument. For example,
Line 83: Line 81:
 
translatable strings and for retrieving the translated values are the same as for internal modules.
 
translatable strings and for retrieving the translated values are the same as for internal modules.
  
See [[Addons_Development#Localization|Addons development]] for more details.
+
See [[Addons_development#Localization|Addons development]] for more details.
 +
 
 +
= Workflow =
 +
 
 +
== Generating the template file ==
 +
 
 +
Strings marked for translation are automatically extracted into a template file called <code>gramps.pot</code>.  This is used to update the <code>.po</code> file for each translation.
 +
 
 +
When translated strings are changed the template file should be regenerated with the following:
 +
 
 +
cd po
 +
python update_po.py -p
 +
 
 +
Weblate will then merge each translation file with the new template and notify translators of new strings to translate.
 +
 
 +
== Adding and removing files ==
 +
 
 +
Every file in Gramps should be listed in either <code>POTFILES.in</code> if it contains translated strings or <code>POTFILES.skip</code> if it doesn't.  These files are used when generating the template file.
 +
 
 +
If a file is added or removed from Gramps, these files should be updated accordingly.
 +
 
 +
== Adding a new language ==
 +
 
 +
To add a new language, navigate to the "gramps" component in Weblate and click on the '''Tools -> Start new translation''' menu option.  Then select a language code.  You need to be a Weblate administrator to do this.
 +
 
 +
Then the language code must be added to the [https://github.com/gramps-project/gramps/blob/master/po/LINGUAS LINGUAS] file.
 +
 
 +
An entry must also be made in the <code>_LOCALE_NAMES</code> dictionary in the [https://github.com/gramps-project/gramps/blob/master/gramps/gen/utils/grampslocale.py#L70 grampslocale.py] file.
 +
 
 +
= Tips for writing a translatable Python module =
 +
 
 +
== Use complete sentences ==
 +
 
 +
Don't build up a sentence from phrases. Because a sentence is ordered in a particular way in your language does not mean that it is ordered the same way in another. Providing the entire sentence as a single unit allows the translator to make a meaningful translation.  Do not concatenate phrases or terms as they will then show up as separate phrases or terms to be translated and the complete sentence may then show up incorrectly, especially in right-to-left languages (Arabic, Hebrew, <abbr title="et cetera - Latin phrase meaning 'and so on'">etc.</abbr>).
 +
 
 +
== Use named %s/%d values ==
 +
 
 +
Python provides a powerful mechanism that allows the reordering of %s values in a string. A translator may need to rearrange the structure of a sentence, and it may not match the order you chose. For example:
 +
print "%s was born in %s" % ('Joe','Toronto')
 +
 
 +
In some languages it may make more sense to say:
 +
print "%s is the city in which %s was born" % ('Toronto', 'Joe')
 +
 
 +
The problem is that this requires a change to the order of the arguments. Python provides a solution for this. By using named operators and dictionaries, we can say:
 +
print "%(male_name)s was born in %(city)s" % {
 +
            'city' : 'Toronto', 'male_name' : 'Joe'}
 +
 
 +
In this case, the order of the %s formatters is not important, since the values will be looked up in the dictionary at run time to resolve the value. The translator can reorder the %s formatters, or even remove them without causing any problems.
 +
 
 +
Note that Python also allows a variation which some people find easier to read:
 +
print "%(male_name)s was born in %(city)s" % dict(
 +
            city = 'Toronto', male_name = 'Joe')
 +
 
 +
Some languages are using right-to-left text direction. It is important to use named arguments when there is more than one %s/%d value into a translation string.
 +
 
 +
== Provide separate strings for masculine and feminine ==
 +
 
 +
Many languages have the concept of gender, while others don't. A sentence may need to be phrased differently  depending on whether the subject is male or female. By using the named %s values along with a bit of code, this problem can be solved.
 +
 
 +
if person.getGender() == Person.male:
 +
        print _("%(male_name)s was born in %(city)s\n") % {
 +
                'male_name' : name, 'city' : city }
 +
else:
 +
        print _("%(female_name)s was born in %(city)s\n") % {
 +
                'female_name' : name, 'city' : city }
 +
 
 +
This allows languages with gender differences to map nicely into your sentence.
 +
 
 +
== Object classes ==
 +
 
 +
Gramps often displays names of primary objects (''Person, Family, Event, <abbr title="et cetera - Latin phrase meaning 'and so on'">etc.</abbr> ...''), for being consistent on displayed strings (also in english!), there is a ''trans_objclass(objclass_str)'' function on TransUtils module.
 +
 
 +
So, when we need to display the primary object name in lower case into a sentence, we can use this function.
 +
 
 +
ex:
 +
from gen.ggettext import sgettext as _
 +
from TransUtils import trans_objclass
 +
 
 +
_("the object|See %s details") % trans_objclass(objclass)
 +
_("the object|Make %s active") % trans_objclass('Person')
 +
 
 +
will display:
 +
 
 +
See ''the person'' details # or See ''the family, the event, <abbr title="et cetera - Latin phrase meaning 'and so on'">etc.</abbr> ...'' details
 +
Make ''the person'' active
 +
 
 +
== Genitive form ==
 +
 
 +
Genitive (and some other) forms need to modify the name itself into some locales, like Finnish or Swedish.
 +
 
 +
Instead of "free form" text that talks about
 +
<abbr title="exempli gratia - Latin phrase meaning 'for example'">e.g.</abbr>,
 +
son '''of %s'''
 +
better would be for example some tabulated format like this:
 +
  son: %s
 +
  daughter: %s
 +
which doesn't require genitive.
 +
 
 +
== Punctuation ==
 +
 
 +
Use of commas, semicolons and spacing can be different than into english.
 +
Remember, simple is better, maybe try to limit punctuation marks.
 +
 
 +
=== definition ===
 +
 
 +
<pre>
 +
$ python3
 +
>>> import string
 +
>>> print(string.punctuation)
 +
!"#$%&'()*+,-./:;<=>[email protected][\]^_`{|}~
 +
</pre>
 +
 
 +
=== locale case ===
 +
 
 +
In French, a space is required before or after some punctuation marks and symbols, like
 +
<pre> : ; « » ! ? % $ # .</pre>
 +
 
 +
* GtkBuilder (editors, configuration dialogs) can provide a default colon after the string without spacing,
 +
so need some extra-testing and customization for some translators. <abbr title="exempli gratia - Latin phrase meaning 'for example'">e.g.</abbr>, in french
 +
 
 +
<pre>
 +
#: ../gramps/gen/plug/report/stdoptions.py:257 ../gramps/gui/configure.py:1222
 +
msgid "Date format"
 +
msgstr "Format des dates "</pre>
 +
 
 +
<pre>
 +
# comté (Canada)
 +
#: ../gramps/gui/configure.py:617
 +
#: ../gramps/gui/editors/displaytabs/addrembedlist.py:75
 +
#: ../gramps/plugins/view/repoview.py:92
 +
msgid "State/County"
 +
msgstr "Province/Comté "</pre>
 +
 
 +
<pre>
 +
# L'espace final est pour précéder le « : » codé en dur.
 +
#: ../gramps/gui/configure.py:1332
 +
msgid "Status bar"
 +
msgstr "Barre d'état "
 +
</pre>
 +
 
 +
== Deferred key on lists ==
 +
 
 +
In most coding situations, strings are translated where they are coded. Occasionally however, you need to mark strings for translation, but defer actual translation until later. A classic example is:
 +
<pre>
 +
animals = ['mollusk',
 +
          'albatross',
 +
          'rat',
 +
          'penguin',
 +
          'python', ]
 +
for a in animals:
 +
    print(a)
 +
</pre>
 +
Here, you want to mark the strings in the animals list as being translatable, but you don’t actually want to translate them until they are printed.
 +
 
 +
Here is one way you can handle this situation:
 +
<pre>
 +
def _(message): return message
 +
 
 +
animals = [_('mollusk'),
 +
          _('albatross'),
 +
          _('rat'),
 +
          _('penguin'),
 +
          _('python'), ]
 +
del _
 +
for a in animals:
 +
    print(_(a))
 +
</pre>
 +
This works because the dummy definition of _() simply returns the string unchanged. And this dummy definition will temporarily override any definition of _() in the built-in namespace (until the del command). Take care, though if you have a previous definition of _() in the local namespace.
 +
 
 +
Note that the second use of _() will not identify “a” as being translatable to the gettext program, because the parameter is not a string literal.
 +
 
 +
Another way to handle this is with the following example:
 +
<pre>
 +
def _T_(message): return message
 +
 
 +
animals = [_T_('mollusk'),
 +
          _T_('albatross'),
 +
          _T_('rat'),
 +
          _T_('penguin'),
 +
          _T_('python'), ]
 +
for a in animals:
 +
    print(_(a))
 +
</pre>
 +
In this case, you are marking translatable strings with the function _T_(), which won’t conflict with any definition of _().
 +
 
 +
See [https://docs.python.org/dev/library/gettext.html#deferred-translations deferred translations]
 +
 
 +
Current custom key on gramps code is '''_T_'''. Set as xgettext flag in [https://github.com/gramps-project/gramps/blob/master/po/update_po.py#L716 <code>update_po.py</code>], used when generating the translation strings template.
 +
 
 +
[[Category:Translators/Categories]]
 +
[[Category:Developers/Tutorials]]
 +
[[Category:Developers/General]]
 +
[[Category:Addons]]
 +
[[Category:Plugins]]

Latest revision as of 02:52, 1 October 2021

Coding guidelines to enable easy and correct translation of strings on the User Interface.

Introduction

Gramps has always been internationalized (see: https://gramps-project.org/blog/2006/04/looking-back-over-5-years/ ) Therefore, all strings meant for the user should always be flagged for translation.

In order to be considered for inclusion in the official Gramps release, any piece of code must support internationalization. What this means is that the Python module must support translations into different languages. Gramps provides support to make this as easy as possible for the developer.

How to allow translations

Gramps is a fully-internationalized application with translations in many languages. All code which presents text to users must provide for that text to be translated. Fortunately, Gramps provides an extension of gettext which makes this fairly painless.

Simple strings

First, alias the gettext function from the single localization instance:

from gramps.gen.const import GRAMPS_LOCALE as glocale
_ = glocale.translation.gettext

This statement imports the gettext function and aliases it as _. The translation tools treat strings wrapped in _() as translatable and assemble them into catalogs for the translators to work with; by aliasing it to gettext(), we also enable python to retrieve the translation appropriate for the user's locale.

Example 1:

print("Hello world!")

In this example, the string will always be printed as specified.

Example 1 internationalized:

print _("Hello world!")

In this example, Gramps will attempt to translate the string. If a translation exists, the call to the function will return the translation. If a translation does not exist, the original string is returned.

Plurals

Plurals are handled differently in various languages. Whilst English or German have a singular and a plural form, other languages like Turkish don't distinguish between plural or singular and there are languages which use different plurals for different numbers, e.g., Polish.

Gramps provides a plural forms support, useful for locales with multiples plurals according to a number (often slavic based languages) or for Asian family languages (singular = plural).

Note, some locales need singular form with zero and plural form might be also used in this case.

In some strings, it's necessary to specify different translations depending upon the number of an argument. For example,

George Smith and Annie Jones have 1 child 
George Smith and Annie Jones have 3 children

We'd code that in python as follows:

_ = glocale.translation.ngettext
_("George Smith and Annie Jones have %(num)d child", "George Smith and Annie Jones have %(num)d children", n) % {num : n}

Context

Sometimes a word in English can have different meanings, for example, "rest" can mean either "what's left" or "take a nap". In such cases, we can provide the context by passing a second optional parameter:

_("rest", "Remaining names")

We're making sure that the translators know the context of the string. The context is made available to the translators in Weblate, but is not displayed to the user.

Comments

A note can be passed to the translators by including a comment in the code on the line before the translated string. This comment must begin with the string "Translators:" and may be multi-line.

# Translators:  Keep your translation short
_("short string")

Glade files

Strings can be marked as translatable in the "Edit Text" dialog in the Glade editor. Context and comments can also be added.

Glade edit text dialog.png

These are saved as translatable, context and comments attributes on an XML element.

<property name="label" translatable="yes" context="context" comments="comment">string</property>

Addons

Third-party addons often need to provide their own message catalogs. To pull one in, use this instead of the usual.

from gramps.gen.const import GRAMPS_LOCALE as glocale
_ = glocale.get_addon_translator(__file__).gettext

or if you need more than one retrieval function:

 _translation = glocale.get_addon_translator(__file__)
 _ = _translation.gettext

The addon translator is another instance of GrampsTranslation, so the rules for creating translatable strings and for retrieving the translated values are the same as for internal modules.

See Addons development for more details.

Workflow

Generating the template file

Strings marked for translation are automatically extracted into a template file called gramps.pot. This is used to update the .po file for each translation.

When translated strings are changed the template file should be regenerated with the following:

cd po
python update_po.py -p

Weblate will then merge each translation file with the new template and notify translators of new strings to translate.

Adding and removing files

Every file in Gramps should be listed in either POTFILES.in if it contains translated strings or POTFILES.skip if it doesn't. These files are used when generating the template file.

If a file is added or removed from Gramps, these files should be updated accordingly.

Adding a new language

To add a new language, navigate to the "gramps" component in Weblate and click on the Tools -> Start new translation menu option. Then select a language code. You need to be a Weblate administrator to do this.

Then the language code must be added to the LINGUAS file.

An entry must also be made in the _LOCALE_NAMES dictionary in the grampslocale.py file.

Tips for writing a translatable Python module

Use complete sentences

Don't build up a sentence from phrases. Because a sentence is ordered in a particular way in your language does not mean that it is ordered the same way in another. Providing the entire sentence as a single unit allows the translator to make a meaningful translation. Do not concatenate phrases or terms as they will then show up as separate phrases or terms to be translated and the complete sentence may then show up incorrectly, especially in right-to-left languages (Arabic, Hebrew, etc.).

Use named %s/%d values

Python provides a powerful mechanism that allows the reordering of %s values in a string. A translator may need to rearrange the structure of a sentence, and it may not match the order you chose. For example:

print "%s was born in %s" % ('Joe','Toronto')

In some languages it may make more sense to say:

print "%s is the city in which %s was born" % ('Toronto', 'Joe')

The problem is that this requires a change to the order of the arguments. Python provides a solution for this. By using named operators and dictionaries, we can say:

print "%(male_name)s was born in %(city)s" % {
           'city' : 'Toronto', 'male_name' : 'Joe'}

In this case, the order of the %s formatters is not important, since the values will be looked up in the dictionary at run time to resolve the value. The translator can reorder the %s formatters, or even remove them without causing any problems.

Note that Python also allows a variation which some people find easier to read:

print "%(male_name)s was born in %(city)s" % dict(
           city = 'Toronto', male_name = 'Joe')

Some languages are using right-to-left text direction. It is important to use named arguments when there is more than one %s/%d value into a translation string.

Provide separate strings for masculine and feminine

Many languages have the concept of gender, while others don't. A sentence may need to be phrased differently depending on whether the subject is male or female. By using the named %s values along with a bit of code, this problem can be solved.

if person.getGender() == Person.male:
       print _("%(male_name)s was born in %(city)s\n") % {
               'male_name' : name, 'city' : city }
else:
       print _("%(female_name)s was born in %(city)s\n") % {
               'female_name' : name, 'city' : city }

This allows languages with gender differences to map nicely into your sentence.

Object classes

Gramps often displays names of primary objects (Person, Family, Event, etc. ...), for being consistent on displayed strings (also in english!), there is a trans_objclass(objclass_str) function on TransUtils module.

So, when we need to display the primary object name in lower case into a sentence, we can use this function.

ex:

from gen.ggettext import sgettext as _
from TransUtils import trans_objclass
_("the object|See %s details") % trans_objclass(objclass)
_("the object|Make %s active") % trans_objclass('Person')

will display:

See the person details # or See the family, the event, etc. ... details
Make the person active

Genitive form

Genitive (and some other) forms need to modify the name itself into some locales, like Finnish or Swedish.

Instead of "free form" text that talks about e.g.,

son of %s

better would be for example some tabulated format like this:

 son: %s
 daughter: %s

which doesn't require genitive.

Punctuation

Use of commas, semicolons and spacing can be different than into english. Remember, simple is better, maybe try to limit punctuation marks.

definition

$ python3
>>> import string
>>> print(string.punctuation)
!"#$%&'()*+,-./:;<=>[email protected][\]^_`{|}~

locale case

In French, a space is required before or after some punctuation marks and symbols, like

 : ; « » ! ? % $ # .
  • GtkBuilder (editors, configuration dialogs) can provide a default colon after the string without spacing,

so need some extra-testing and customization for some translators. e.g., in french

#: ../gramps/gen/plug/report/stdoptions.py:257 ../gramps/gui/configure.py:1222
msgid "Date format"
msgstr "Format des dates "
# comté (Canada)
#: ../gramps/gui/configure.py:617
#: ../gramps/gui/editors/displaytabs/addrembedlist.py:75
#: ../gramps/plugins/view/repoview.py:92
msgid "State/County"
msgstr "Province/Comté "
# L'espace final est pour précéder le « : » codé en dur.
#: ../gramps/gui/configure.py:1332
msgid "Status bar"
msgstr "Barre d'état "

Deferred key on lists

In most coding situations, strings are translated where they are coded. Occasionally however, you need to mark strings for translation, but defer actual translation until later. A classic example is:

animals = ['mollusk',
           'albatross',
           'rat',
           'penguin',
           'python', ]
for a in animals:
    print(a)

Here, you want to mark the strings in the animals list as being translatable, but you don’t actually want to translate them until they are printed.

Here is one way you can handle this situation:

def _(message): return message

animals = [_('mollusk'),
           _('albatross'),
           _('rat'),
           _('penguin'),
           _('python'), ]
del _
for a in animals:
    print(_(a))

This works because the dummy definition of _() simply returns the string unchanged. And this dummy definition will temporarily override any definition of _() in the built-in namespace (until the del command). Take care, though if you have a previous definition of _() in the local namespace.

Note that the second use of _() will not identify “a” as being translatable to the gettext program, because the parameter is not a string literal.

Another way to handle this is with the following example:

def _T_(message): return message

animals = [_T_('mollusk'),
           _T_('albatross'),
           _T_('rat'),
           _T_('penguin'),
           _T_('python'), ]
for a in animals:
    print(_(a))

In this case, you are marking translatable strings with the function _T_(), which won’t conflict with any definition of _().

See deferred translations

Current custom key on gramps code is _T_. Set as xgettext flag in update_po.py, used when generating the translation strings template.