15,534
edits
Changes
→Advanced usage
There are versions for 3.1, 3.2, 3.3, 3.4.x and 4.1.x as well as an old version that works with Gramps version 3.0.x or 2.2.5+ ! [[Category:PluginsPlace_completion_tool#Download|Download]].
== Place Completion tool ==
The general aims are:
*Place/Location is a newer concept in GRAMPSGramps. Many older databases only have a Place title field which is a descriptive text containing city, state, country. This should be parsed to insert the values in the correct attribute fields.
*Latitude and longitude are important to show data on a map. However, doing a look-up of this data on the internet is slow and time consuming. The tool allows to search in the free resources on the net.
*Setting of an attribute of a set of places in one go. Eg you give a *Conversion of latitude and longitude to a fixed data format. On import one might obtain latitude and longitude in several different formats. A conversion tool to store them all in the same format is usefulluseful.*Construction of a uniform title/description field, from the data in the place object == Design Specification ==See [[Place completion tool specification]]Place/Location is a newer concept in GRAMPS. Many older databases only have a Place title field which is a descriptive text containing city, state, country. To distribute these values into the correct attribute fields, see Parse Title Details below
== Manual ==
=== Download resources ===
The place completion tool can look up for you latitude/longitude, add county information (USA), ... . For some of this functionality, you must download datafiles data files of the countries you are interested in. Right now you have three options:
#Download geonames country files. You can do this [http://download.geonames.org/export/dump/ here freely]. Geonames parses fastest, so is the advised format to use
#Download geonames USA state files. You can do this [http://geonames.usgs.gov/domestic/download_data.htm here freely]. This is advised for USA searches, as the data in the USA country file contains many doubles, which can be avoided by searching state per state. State info also contains county information.
#download GNS Geonet country files (not available for usa). You can do this [ftp://ftp.nga.mil/pubpub2/gns_data here freely with ftp].
Watch out, some of these downloads are '''VERY''' large, especially USA data. Only download what you need!If the download is a compressed zip file, you will need to extract the data file before you can use it.
<small>'''Note''': The geonames data of popular places is in English, so eg e.g. municipalities in Italy will be found, but Roma not, as this is Rome in English. To find data with these you need to search in the localised variants of the name (see below)</small>
{{man warn|Warning:|DO NOT BETA TEST WITH YOUR RESEARCH DATA. EXPORT DATA FIRST TO HAVE A BACKUP.}}
=== Starting the tool ===
You will find the plugin under 'Tools > Utilities > Interactive place completionPlaceCompletion'
=== The dialog explained ===
The Dialog consists of 4 parts:
====Part 1: selection . Selection of places====
First you need to choose with which places you want to work. You can use several methods to define your places:
# Use a place filter. You can use two preset filters: ''All places'', which returns all places, and ''No Latitude/Longitude given'', which returns all places of which the latitude or the longitude is not set. You can also created a custom place filter in the place view, test it with the filter sidebar, and then use it in this tool. All custom filters you made will be available
# Use a latitude, longitude rectangle. Eg, suppose you have the latitude and longitude of all places in the UK, and now want to add in the state attribute ''Wales'', for all places in Wales. You can look on a map, note down the centre of Wales in latitude and longitude, as well as roughly the width and height of this rectangle. This will allow you to obtain all places in Wales (and some in England), allowing to much faster set the state information.
====Part 2. Completion of Places places ====
#The first possibility is to look up in a datafile the latitude and longitude of your places. For this you must have downloaded the necessary resources, see section above. You can select with a file dialog the file you want to search, and set how this data must be parsed. The following parsing options are available:
##''GeoNames country file, city search'': use the city attribute to look for lat/lon in a GeoNames country file. This is the fastest search.
###All in degree notation: use the classical degree notation with degree, minutes and seconds.
###All in decimal notation: use the decimal system to denote lat/lon.
###Correct -50° in 50°S: a much seen error is to use - for the classical degree notation, which is wrong, and which GRAMPS Gramps will not be able to interpret. With this option this error is looked for and corrected.
# A third option is to set attributes of all selected places. You can set the country, state, county, parish, zip/postal code and city attributes of all places in one sweep.
====Part 3. Overview of the results ====
After having entered all data in Part 1 and 2, you click find for GRAMPS Gramps to search all changes that will occur. This part of the dialog shows all changes that will occur.
[[Image:place_completion_tool_results.png]]
== Example ==
Open the example file from the examples where latitude and longitude are empty: [http://grampssvn.svncode.sourceforgesf.net/viewvcp/gramps/*checkout*code/grampsbranches/trunkmaintenance/gramps2gramps34/example/gramps/example.gramps?revision=7186 example.gramps].
We will now show how the places in this file can be completed. The best thing to do is open a new family tree (.grdb), give it a name, and import the example.gramps file. This file has 852 places, which would mean a lot of manual edits if you do not use this tool!
In case google maps did not allow you to determine which is the correct city, you can double click on the city to open the Place Dialog ('''Warning: this will preenter the data of the Place Completion tool. So hit cancel here if you want to exit without these changes done'''). In this dialog the references tab allows you to navigate to all events coupled to this place. This will give you extra information you might use to decide which of the two found places is the correct place.
=== Step 3: Problem entries===
While updating all places in step 2, you will have noticed some errors in the state information: Some places have a dubious state: eg OH-AL
You can obtain these states by choosing ''All Places'' en setting the state search box to '''-'''. Clicking Find will give you all these problem places. You can use google maps or the place dialog to sort them out. You can also use the USA country GeoNames file to search these places in the entire USA. '''Note:You will need sufficient memory for this, or you will obtain a MemoryError (see below)!'''
=== Step 4: Lat/Lon not found ===
It will be clear that many of those can be quickly corrected: abbreviations, eg the city field contains ''St.George'', which should be ''Saint George''; double names, eg Waterloo-Cedar Falls, IA means Waterloo near Cedar Falls, changing the city to Waterloo and redoing the search using Google Maps will allow to quickly find which coordinates for Waterloo are needed.
== Advanced Usage usage == This is for advanced users only knowing who know how to use [http://en.wikipedia.org/wiki/Regular_expression regular expressions].
The parsing fields have entry fields allowing you to give your own parsing. Parsing uses regular expressions. You can use this to parse your title, and to parse a lat/lon file in your own way. For reference, here an overview of the parsing codes used for the predefined parses:
=== Parse title details ===
In 'Parsing and Conversion of existing title or position', 'Parse title' and 'Change title into' provide some pre-defined options for extracting location values from a Place Title. Otherwise regex parsing is needed.
Write your regex in 'Parse title:'.Click on 'Find', which shows the proposed changes. Then click 'Apply'.
'''Example 1''' For France, some [http://www.geneawiki.com/index.php/Informatique_-_saisie_des_lieux practical rules] could be useful for seizing place. It needs : * the city name + [http://en.wikipedia.org/wiki/INSEE_code#Geographical_codes INSEE code] (at option). This code is unique and can identify with certainty a common (with the county, district, township and municipality). It can identify with a common insurance even if it has changed its name. This code is used in Archives. ''Using postal code is not advisable ...'' * a subdivision: identifies a parish or a place called within a municipality * the state (at option) ''or county but is already on [http://en.wikipedia.org/wiki/INSEE_code#Geographical_codes INSEE code]'' * the country (at option). Ideally it should still take the country. It is understandable that this is tedious. ''Maybe do not enter the country if the genealogy is mostly of one country and seize enter the country for events outside the country's main. Everyone will appreciate.'' e.g. Avignon,84000,Vaucluse,Provence-Cote-d'Azur,FRANCE, where some fields may be missing: e.g. Woerth,,,Alsace,FRANCE, the regex: <code>\s*(?P<city>[^,]+)[,]\s*(?P<zip>\d*)[,](?P<county>[^,]*)[,](?P<state>[^,]*)[,](?P<country>[^,]*)[,]*$</code> treating the comma character as an end-of-string delimiter, will distributeAvignon to City, 84000 to ZIP, Vaucluse to County, Province-Cote-d'Azur to State and France to Country and in the case of missing fields, as long as the first is not empty, will distribute Woerth to City, Alsace to State and FRANCE to Country. It allows initial whitespace and an optional comma after the Country '''Example 2''' In many 16th and 17th century English IGI records the situation is more complicated. The Place Title takes the form of 3 strings (Town, County, Country) or 4 strings (Parish, District, County, Country) for example:
(a) Chester le Street, Durham, England - 3-string
Regex A:
<code>\s*(Of[,]*\s*)*(?P<city>[^,]+?)[,]\s*(?P<county>[^,]+?)[,]\s*((?P<country>[^,]+?)){1,1}$</code>
Regex B:
<code>\s*(Of[,]*\s*)*(?P<parish>[^,]+?)[,]\s*(?P<city>[^,]+?)[,]\s*(?P<county>[^,]+?)[,]\s*((?P<country>[^,]+?)){1,1}$</code>
To avoid this, when using Regex B click on 'Find' to display the records proposed for change and delete all the 3-string 'Of,' records before clicking 'Apply'.
{{man note|take care when pasting a Regex (for example, after testing in a regex editor) into the Parse title details field. Pasting adds redundant spaces before or after the Regex that will prevent it working properly in the Place Completion Tool}}
The predefined regex expressions are as follows, where for brevity we use some variables defined lower.
Resource: [http://de.wikipedia.org/wiki/Wikipedia:GEOnet_Names_Server GEOnet]
== Design specification ==
*See [[Place completion tool specification]]
Place/Location is a newer concept in Gramps. Many older databases only have a Place title field which is a descriptive text containing city, state, country. To distribute these values into the correct attribute fields, see Parse Title Details below
== Troubleshooting ==
MemoryError</nowiki>
The tool has to load the datafile for latitude/longitude searching into memory. For large files like USA.txt, this might be impossible if you have limited memory. You can try to close as many programs running at together with GRAMPSGramps, and try the tool again.
== Download ==
If you still use Gramps 32.2 (trunk).5+, then you will need to make a couple version 1.0 of changes to the Place Completion Tool. You find it at [http://cage.ugent.be/~bm/varia/placecompletion_1_2placecompletion_1_0.tar.gz placecompletion_1_2placecompletion_1_0.tar.gz]. For example, here is a [[Media:PlaceCompletion.patch|patch]] that will allow it to work.
If you still use Gramps 3.10.x, then you will need version 1.2 1 of the Place Completion Tool. You can find it at [http://cage.ugent.be/~bm/varia/placecompletion_1_2placecompletion_1_1.tar.gz placecompletion_1_2placecompletion_1_1.tar.gz].
If you still use Gramps 3.01.x, then you will need version 1.1 2 of the Place Completion Tool. You can find it at [http://cage.ugent.be/~bm/varia/placecompletion_1_1placecompletion_1_2.tar.gz placecompletion_1_1placecompletion_1_2.tar.gz].
* local install: place in <code>~/.gramps/plugins</code>
* global install: <code>place_to_gramps_install/src/plugins</code>
