Date Handler

From Gramps
Jump to: navigation, search

Why have different date handlers?

Different cultures and regions tend to have specific and different conventions for parsing and displaying dates. For example, the month and day order is different between most european coutries and the US. Also, each language has its own set of acceptable modifiers and qualifiers for the date: things like "from X to Y" or "between X and Y" may have different word order in different languages. Same with "around", "calculated", "estimated". Add to this calendar names and you have a compelling need for a dedicated date handler.

By providing date handlers, several problems are resolved.

  • Dates entered by users of that language in almost any form can be parsed by Gramps.
  • The displayed dates will look clear and correct to the users.
  • Translators do not have to worry about translating regular expressions like (from|before|between).

Unfortunately, it turned out that the maintenance burden of date handlers across all the languages is very high compared to the regular maintenance overhead of strings in the translation database. Under 6926 we attempted to shift the balance back from coding to translation database. The new system also allows to support inflections for month names in the languages where it is needed. Date handler code still exists, but is much simpler and easier to maintain. See also the related open issues.

How to write a date handler

Gnome-important.png
Since new code for Gramps is developed on trunk, the present instructions only refer to the current API. If you are interested in older API documentation, see the revision history on the wiki and in the source control.

Please use the generated documentation on the date handler framework as your programming reference. Here are the some highlights on how a date handler plugin for a new language should be developed.

  • The plugin must translate some strings in the translation database. Special conventions allow for finer linguistic control for inflected forms of month names for languages where this is needed.
  • The plugin must define two classes: one for parsing and one for displaying dates. Various degrees of customizations exist, depending on how different the standards for formatting the dates in your locale are from English.
  • The plugin should be registered in the system.
  • Before submitting it to the code base, it should be tested using the existing testing framework.

Localizing the date strings

Use the regular translation routine to translate the strings such as month names, date modifiers etc. These strings are stored in the class DateStrings, which is used both from the date parser and the date displayer.

If your language has special inflection rules, note that these strings are processed with gen.utils.grampslocale.GrampsTranslation.lexgettext into instances of gen.utils.grampslocale.Lexeme. Use the examples in the Lexeme docs to learn how to provide all the inflections you need for all the contexts used later in the date displayer.

For example (from ru.po):

#: ../gramps/gen/datehandler/_datestrings.py:70
msgid "localized lexeme inflections||May"
msgstr "И=май|Р=мая|Т=маем|П=мае"

All the provided inflections and all their unambiguous prefixes will also be automatically recognized as valid synonyms for the date parser base class code.

Providing a localized date displayer class

The displayer class must derive from the DateDisplay class:

   from _DateDisplay import DateDisplay
   class MyDateParser(DateDisplay):
       ...

The displayer class must provide display() method.

If you inherit the default implementation, then all the dates will be displayed in iso style, without any visible difference between span and range dates.

The next level of customization is to select another pre-canned alternative, provided in the base class, as follows (like DateDisplayEN does):

       display = DateDisplay.display_formatted

This will be sufficient for over half of the currently supported languages, that don't tweak the list of formats available for date display.

The core of display_formatted displays both regular and compound dates, and adorns them, if necessary, with modifiers (as in "before..."), quality indicators (as in "calculated..."), and calendar/non-standard year start information.

If your language requires inflections of month names dependent on modifiers, or, for instance, whether the date is a start or a stop date in a range, there are special strings you need to translate to indicate the appropriate inflection. Documentation for this advanced scenario can be found in the source code comments to DateDisplay.

If your language overrides Display.formats to something else, it should also override _display_calendar (or, separately, the _display_gregorian/_display_hebrew/... whatever methods that delegate to _display_calendar by default with the appropriate month lists). The override of _display_calendar should format the date according to the currently selected custom format, using self.format number as an index into the self.formats list.

Providing a localized date parser class

The parser class must derive from the DateParser class:

   from _DateParser import DateParser
   class MyDateParser(DateParser):
       ...
The parser class must provide parse() method. In fact, since the base class already defines such method, it is most likely that you will only need to re-define class constants and, maybe, the init_strings() method.

Localizing the date formats

If you use DateDisplay._display_calendar in your date displayer, whether directly or via DateDisplay.display_formatted, then your dates will be formatted using the translated format strings therein. The month names (whether short or long) in these formats are used to format an instance of the class Lexeme, initialized to the set of inflection forms from the translated month string.

These formats all have a warning in the translation database sending you to the entrance point to these docs. You have the following options in translating such a format string:

1) If you don't translate the string, it will just use the same order as in English, without any inflection selection.

2) If you omit the inflection selection in a format, then the default inflection will be used, or, if your language has no inflections, just the one and only name for the month (see the documentation for class Lexeme for details). REMEMBER the format specifiers themselves must not be translated, otherwise there will be a runtime error in the formatting code. E.g.:

msgid "{day:d} {long_month} {year}"
msgstr "{day:d}/{long_month}/{year}"

(Note how the words "long_month" remain in English!!!)

3) However, if your language does support inflections, here you should select the appropriate inflection for the month name, using the same name of the inflection as you entered in the translated month string. Again, see the documentation for class Lexeme for details. E.g.:

#. TRANSLATORS: see
#. http://gramps-project.org/wiki/index.php?title=Translating_Gramps#Translating_dates
#. to learn how to select proper inflection for your language.
#: ../gramps/gen/datehandler/_datedisplay.py:475
msgid "{day:d} {long_month} {year}"
msgstr "{day:d} {long_month.f[Р]} {year}"

Some of the strings need not be translated at all for your language if you don't use the inflection control feature of _display_calendar, e.g.:

#. first date in a span
#. You only need to translate this string if you translate one of the
#. inflect=_("...") with "from"
#: ../gramps/gen/datehandler/_datedisplay.py:160
msgid "from|{long_month} {year}"
msgstr ""

Documentation for this advanced scenario can be found in the source code comments to DateDisplay.

Upgrading an older date handler to the framework from 6926

First, before changing your code, run the helper code to generate a snippet you can merge into your language's xx.po. See the documentation.

If you stop here, the handler should work already (strings previously in _grampslocale.py are now picked up from the translation database, and the rest remains the same). However, it'll likely have lots of redundant code and strings in it...

Then remove from your DateDisplayXX class any overrides of date strings from base class, such as long_months, short_months, calendar, _mod_str, _qual_str -- these are now localized via the translation database.

NOTE: Leave _bce_str for now (it will be removed when 7064 is resolved).

In your DateParserXX class, remove the various ..._to_int that are now initialized based on the strings in DateStrings, i.e.:

month_to_int
hebrew_to_int
french_to_int
islamic_to_int
persian_to_int
calendar_to_int

If you had various spellings of the same word, you can use the lexeme mechanism to express that in the translation database:

#: ../gramps/gen/datehandler/_datestrings.py:140
msgid "Hebrew month lexeme|Nisan"
msgstr "И=нисан|Р=нисана|Т=нисаном|П=нисане|И1=ниссан|Р1=ниссана|Т1=ниссаном"

Here we unite the alternative spellings Nisan and Nissan into the same lexeme.

Alternatively, you can extend the base string table that is initialized from the date strings explicitly from the code, as before. For example, here's how the Russian parser adds an alternative name for the Persian calendar:

   def init_strings(self):
       DateParser.init_strings(self)
       DateParser.calendar_to_int.update({
           'персидский'    : Date.CAL_PERSIAN, 
           'п'             : Date.CAL_PERSIAN, 
       })
       ...

If you don't have custom language-specific formats

If you override formats and it corresponds exactly to the formats in the DateDisplay.formats, only changing the strings inside to translate them to your language, then remove the formats override and any overrides of _display_gregorian/_display_julian/... and display methods.

Just alias your display to DateDisplay.display_formatted.

If you have custom language-specific formats

Consider rewriting your existing overrides of _display_gregorian and display to a single override of _display_calendar. At the very least, you'll need to update the signature of your _display_* methods, to match _display_calendar. You may just add (and ignore) a trailing **kwargs if you're lazy...

Converted handlers

Add yours here when done!!!

  • FR (romjerome)
  • HR (josip)
  • RU (vassilii)

Registering the localized date handler

The module must register itself as the date handler. This is done by inserting the following code at the end of your _Date_xx.py file:

   from _DateHandler import register_datehandler
   register_datehandler(('xx_XX','xx','xxxxx','xx_YY'),
                        MyDateParser,MyDateDisplay)
where MyDateParser and MyDateDisplay are the classes defined for parsing and displaying, and the items in quotes are language identifiers that may possibly be associated with your language. For example, different systems use ru, RU, ru_RU, koi8r, ru_koi8r, russian, Russian, ru_RU.koi8r, etc. to identify the Russian language.

Then you need to import the parser to Gramps by adding

   import _Date_xx

line to __init__.py file located in the same directory.

That's it for the requirements. The example date handling plugins can be found in _Date_xx.py under the gramps/gen/datehandler directory.

Classes and relations

Gnome-important.png
not updated yet -- any way to ask Sphinx to do it?

A quick overview of current classes and relations between Date handlers.

Calendars

Gramps uses Gregorian calendar by-default, but also supports Julian calendar, Hebrew calendar, Islamic calendar, Persian calendar, French Republican calendar and Swedish calendar.

It is possible to add new one, like Chinese calendar, Indian Civil calendar, Bahá'í calendar, Mayan calendar, Zoroastrian calendar or Japanese calendar ... into gen/lib/calendar.

We need to have rules which properly convert this new calendar to sdn (serial date number) and from sdn to calendar.

How to test a date handler for your locale

Testing with Gramps

When you run from the Git tree, you have the dateparserdisplaytest.py debug tool available under Menu -> Tools -> Debug ->.

  1. Create a new empty Family Tree.
  2. Go on Menu -> Tools -> Debug -> test for date parser and displayer.
  3. Compare source and target dates in the entries on Events View. Alternatively, compare the birth and death dates in the People View. In the latter view, the failed tests are marked with a "Fail" tag on the person, you can use the "Customize view" option to add the "Tags" column to sort on the pass/fail status.

Tests with command-line

In gramps/ directory:

1) Ensure you haven't broken the basic date functionality in English:

 LC_ALL=en_GB.utf-8 LANG=en_GB.utf-8 GRAMPS_RESOURCES=$PWD  python3 -m unittest discover -p 'date*test.py'

2) Test your locale. For example, for Russian:

 LC_ALL=ru_RU.utf-8 LANG=ru_RU.utf-8 GRAMPS_RESOURCES=$PWD  python3 -m unittest discover -p 'date*test.py'

See also Testing Gramps.

See also

Known bugs

  • 7084 576/3795 DateParserEN failures under the DateTest tool