GEPS 045: Place Model Enhancements

From Gramps
Revision as of 22:22, 6 June 2019 by Prculley (talk | contribs) (Recommendation)
Jump to: navigation, search

Rethinking Gramps Place data model

One of the items on the Gramps 5.1 Roadmap is to look into better GEDCOM support for places. We could continue to force fit GEDCOM import data to Gramps, or look into expanding Gramps data model to make for a better fit.

Place Enhancements

Place.Type:

Gramps only supports a single place type. Gov (http://gov.genealogy.net/search/index) and GEDCOM L group support multiple types, each with a subordinate date range, GEDCOM L group support a citation for each type as well

Recommendation

Make place types a list. Each element should have a subordinate date range and will have a citation list. (done)

We should also provide a routine similar to the current place displayer to return a place type for a particular date. (done, gen.utils.location.__get_type)

Place Types:

Changes to place types, do not actually modify the data model, but dealing with them better will certainly impact a lot of code.

GeoNames currently doesn’t do much to provide place types. It currently uses PCLx, for Country, PPLx for 'Populated Place', and ADM1-ADMn for different level of administrative place. These do not map well to Gramps PlaceTypes. The Place Cleanup Gramplet tries to do it anyway, but allows user modification.

Gov currently has over 260 place types in several broad categories. The categories are Administrative, Civil, Religious (church), Judicial, populated place, transportation, and a few miscellaneous others. They are still adding more. The GetGov Gramplet currently stores the returned German place type as a Custom Gramps place type, which is of course not translatable. For example, Deutsches Reich, or USA gets a PlaceType of 'Staat', rather than country. While the system has a lot of types it is very German centric, many types have only German names in their definition documents although some of the more commonly used types have at least English translations and some have several other translations.

Recommendation

One of;

  1. ) Adopt the Gov type system. We would have to do a lot of translation work on many types, or indicate that the type is local to certain countries (so could use the local words). Our GUI would probably have to use at least a two level hierarchy in the PlaceType selector, possibly three levels, to help users find appropriate types for manually entered places or filtering.
    One of the levels (top level?) would be the category type.
    An additional level might be the country specific types.
  2. ) Provide a conversion of the Gov type to the closest Gramps PlaceType, possibly adding a few more Gramps PlaceTypes. If we did that, I think it would be reasonable to store the original Gov type in a place fact.
  3. ) Keep our standard Place Types, allow GOV types to be added as custom types as needed.
    Add a concept of Place type groups, which make it possible for multiple types to be identified as Country (for example). The Groups are G_COUNTRY, G_REGION, G_PLACE, G_UNPOP, G_BUILDING.
    The GUI type selector is enhanced to support additional levels corresponding to the groups (only when needed).
    The Edit/Preferences/Place Types (new panel) is added to allow editing of the place type names, addition and removal of place types, and assignment to groups. It is also possible to add and remove groups.


The third option is accepted for development. (done)

Place.Name:

Gramps supports multiple names, each with a language and date range.

GEDCOM supports two additional parts of a name; neither of which seem to be used much and are not currently supported by Gramps.

FONE (Place phonetic variation) and its subordinate TYPE. This seems to be used for far eastern places for example if hiragana was used to provide a reading of a name written in kanji, then the <PHONETIC_TYPE> value would indicate kana. The TYPE is defined as [<user defined> | hangul | kana]

ROMN (PLACE_ROMANIZED_VARIATION:= {Size=1:120}) and its subordinate TYPE. The romanized variation of the place name is written in the same form prescribed for the place name used in the superior <PLACE_NAME> context. The method used to romanize the name is indicated by the line_value of the subordinate <ROMANIZED_TYPE>, for example if romaji was used to provide a reading of a place name written in kanji, then the <ROMANIZED_TYPE> subordinate to the ROMN tag would indicate ‘romaji’. The TYPE is defined as [<user defined> | pinyin | romaji | wadegiles]

GEDCOM L group adds in _NAMC (place name addition) for each name. Their documentation currently says "Addition to PLACE_NAME, can be appended to PLACE_NAME in non-delimited reports to make the label unique". This is for suffixes like "am Main" in "Frankfurt am Main".

GEDCOM L group adds in ABBR with a subordinate TYPE. ABBREVIATION_OF_NAME Abbreviation for the place name, whereby the type of abbreviation can be further explained with the optional TYPE.

GEDCOM L group adds in a citation list for each name.

The GeoNames data and Place Cleanup Gramplet currently store place name abbreviations as an alternate name with the 'abbr' language code.

The GeoNames data and Place Cleanup Gramplet currently store colloquial place names ('Big Apple' for New York City USA) as an alternate name with no language code.

Recommendation

  1. ) Our primary name and alternate names become part of a more general place name list. Each has a date, language, and a citation list associated with it.
  2. ) Add an abbreviation list to each place name. Each abbreviation has a type and value. In Reports and graphs, when abbreviations were specified to be used, the language/date specific name abbreviation would be used instead by the place displayer.
  3. ) For place name abbreviations, the place display format editor allows selection of the abbreviation by type of first available in the list for a particular name. If no abbreviation is present, use a place displayer routine uses the full name.

Done

Postal Code:

Gramps currently offers a single field 'Code' to store postal codes. It may have also been used for other purposes in the past.

GeoNames, GOV, GEDCOM L all support multiple postal codes. GEDCOM L adds in a subordinate DATE range and citation for each code.

The Place Cleanup Gramplet currently stores multiple postal codes in the code field separated by commas.

Recommendation

One of the following:

  1. ) Make the postal code(s) part of a more general place attribute list. Each code (attribute) a citation list associated with it. Since an attribute doesn't have a date, the date would be appended as text to the attribute value (Ex: "12345; Date: JAN 12 2019)


Done

MAIDENHEAD_LOCATOR

GEDCOM L has added this to the Place import/export.

The Maidenhead Locator divides the world into small areas (micro-fields) marked by 8 letters and numbers. It was originally conceived for use in ham radio, but it is also used as a geographic locator in some other applications. https://en.wikipedia.org/wiki/Maidenhead_Locator_System

Recommendation

Store the Maidenhead locator in the place attribute list. Date and Citation parts would not be used. (Done)

ADMINISTRATIVE_IDENTIFIER

GEDCOM L has added this to the Place import/export.

The official or public identifier for a location object, such as community code, ISO 3166 for countries and states. GEDCOM L allows multiple of these, each with date range and an additional TYPE and citation.

Recommendation

Store the ADMINISTRATIVE IDENTIFIER in a more general place fact list along with the Date and Citation. Merge the TYPE_OF_ADMINISTRATIVE_IDENTIFIER into the fact type.

DEMOGRAPHIC DATA

GEDCOM L has added this to the Place import/export. Demographics about people to the place object, such as number of households, number of people, occupations, and so on. GEDCOM L allows multiple of these, each with date range and an additional TYPE and citation.

Recommendation

Store the DEMOGRAPICAL_DATA in a more general place fact list along with the Date and Citation. Merge the TYPE_OF_DEMOGRAPICAL_DATA into the fact type.

EVENT_DETAIL

GEDCOM L has added this to the Place import/export.

GEDCOM L has added an event reference to the place.

Recommendation

Add an Event reference list to the Place. The list could contain one of the following:

  1. ) EventRef; The Event ref role would be set to 'Place'
  2. ) Direct pointer to Event object. (My preference)

Enclosed by

The following items would be part of the Gramps place reference, which currently only includes a date and the reference to another place.

HIERARCHICAL_RELATIONSHIP

GEDCOM L has added this to the Place import/export.

Values are [POLI | RELI | GEOG | CULT] to differentiate political (administrative), ecclesiastical, geographical or cultural attributions. GEDCOM L includes this to provide some indication of the type of enclosure. For example a Church might be enclosed by a city and enclosed by a parish, with different relationships.

Citation

GEDCOM L adds a citation to the Enclosed by

Recommendation

No change. The Hierarchical relationship can be inferred from the type of the enclosing place, so it appears to me to be redundant. I also think that a citation here adds little value.

This presumes that Place Types are extended to include additional types in multiple categories.

Place facts

To deal with several new facts associated with places, I propose a generalized Place Facts list. Each place fact would include the type, value, date, and possibly citation.

Citations

GEDCOM L has added citations to most of its additional data. To me this seems excessive. I think we need to decide if we support citations individually, or just citations on the Place object. This would affect the proposed Place Fact structure, as well as Place Name, Type, Postal codes, and Place Enclosed by.

ToDo

Top priority

If we do these first and carefully, we should be able to run Gramps with new trees. gen.lib.place

  • update object, get_schema, serialize/unserialize etc.
  • Add add_eventref, get/set/merge_eventreflist
  • Add add/get/set/merge_place_facts
  • add_place_type, get/set/merge_place_types. Consider keeping the most modern place type (based on date) with admin hierarchy at top of list (makes views.placemodel a bit faster to pick type to view and allows for get_place_type to return top value for short term compatibility).
  • Remove get/set title (or covert to use place.displayer)
  • Get/set_name – Do we want to have this separate, or as just top of a list of names?
  • Add get/set/merge_abbreviation
  • Remove or modify get/set_code (if modify, set/return top postal code fact)
  • Remove get/set/merge_alternative_locations (Note: need to move to notes for upgrade)
  • fix up other methods as needed.
  • Do we add citationrefs to place names, types, placerefs?

gen.lib.placefact – new. Similar to gen.lib.attribute but with date. gen.lib.placefacttype – new. Similar to lib.eventtype, but for place facts. gen.lib.placetype – update for more place types and hierarchy gui.views.placemodel – update for changed serialization. Do we still want to display 'Code'? Propose not (if not, also change plugins.lib.libplaceview for missing column).

2nd priority

Editors

I'm assuming that the place editor is not significantly changed in style from the current one.

  • Add place type editor icon to place type? Instead of combo.
  • Place type editor gui.editors.editplacetype? Combo for type and Add date and citationref
  • Add place Types tab gui.editors.displaytabs.placetypeembeddedlist and placetypemodel
  • Remove Title edit capability
  • Remove Code (data is moved to place facts tab).
  • Add Place Facts tab gui.editors.displaytabs.placefactembeddedlist and placefactsmodel
  • Add place Events tab - gui.editors.displaytabs.placeeventembeddedlist and placeeventmodel
  • Duplicate place editor work for gui.editors.editplaceref
  • gui.editors.editevent – fix DeleteEventQuery to scan for places containing events

Views

  • Add Place Facts and Events Gramplets to Place views plugins.lib.libplaceview
  • Modify Events Gramplet plugins.gramplets.events to deal with place events (remove age column).
  • Add Place Facts Gramplet plugins.gramplets.placefacts similar to Attributes Gramplet but with dates

Import/export/upgrade

  • Update dtd
  • Update XML export
  • Update XML import
  • Modify bsddb update routines plugins.db.bsddb.upgrade
  • Create dbapi update routine plugins.db.pbapi.upgrade, and hooks into db loaders

DB upgrade or XML import should:

  • put legacy place title into a Note, if it differs from the place displayer.
  • Move Code field into place fact
  • Move alternate Location into Note or place Fact

Merge

  • Fix gen.merge.mergeeventquery to check for place events

3rd priority

Import/Export

  • Update Gedcom export from GEPS 043 work to support place model https://github.com/gramps-project/gramps/pull/469
  • Update importer for place changes; csv
  • Update importer for place changes; geneweb
  • Update importer for place changes; progen
  • Update importer for place changes;vcard
  • Update exporter for place changes; csv
  • Update exporter for place changes;ftree
  • Update exporter for place changes;geneweb
  • Update exporter for place changes;vcalendar
  • Update exporter for place changes;vcard

Reports

  • Place Format Editor?
  • Narweb, no obvious changes.
  • Plugins.graph.gvfamilylines – Uses_get_main_location, may want to update to use place abbreviation.
  • Plugins.graph.gvrelgraph – Uses_place displayer, may want to update to use place abbreviation.
  • Plugins.libllibstubkeyword – May want to rethink this, at a minimum include the place abbreviation as a choice. Uses get_main_location…
  • Plugins.lib.maps.geography; add_place_from_kml needs minor update
  • Plugins.lib.maps.placeselection may need to do something with get_location, depends on specific place types.
  • Plugins.mapservices.eniroswedenmap uses some specific place types…
  • Plugins.textreport.placereport – may want to fill out data with new stuff

Tools

  • Generate Testcases may want to add some additional place items
  • Check & Repair; check_events needs to look at places for references.

Filters

  • Gen.filters.rules.has_data place type scan should look at whole list
  • New filter rules for place facts, place events
  • Place Sidebar filter; remove code, maybe add fact filter

Addons

  • CheckPlaceTitles – don't need this anymore?
  • DataEntryGramplet – has set_title for a place
  • DescendantBookReport – has get_title for a place
  • DetailedDescendantBookReport– has get_title for a place
  • DescendantsLines\substkw – same fixes as for Plugins.libllibstubkeyword, or just use the plugins.lib version.
  • DynamicWeb – uses place.get_title place get_main_location, get_alternate_locations, get_code
  • ExtractCity – uses set_code
  • GetGov – modify to store data in correct parts of place, adapt to updated Place Types with hierarchy and date.
  • GraphView – may want to add support for place abbreviation
  • ImportMerge – make sure algorithm doesn't break on Event/Place/Event loop
  • PlaceCleanup – Need to store data in correct parts of place, change Code handling, change abbreviation handling, store dated facts, multiple place types etc.
  • PlaceCompletion – don't support this anymore?
  • Sqlite.ExportSql – needs update if we want to support
  • Sqlite.ImportSql – needs update if we want to support

Documentation

  • Update Data model diagram
  • Update wiki (entering data, place name editor, …)

Test code

  • Db tests
  • Import/export tests
  • Filter tests

Comments

See also