Difference between revisions of "Addons development"

From Gramps
Jump to: navigation, search
(Test your addon as you develop)
(5 intermediate revisions by 2 users not shown)
Line 90: Line 90:
  
 
=== Other pre-requisites ===
 
=== Other pre-requisites ===
 +
{{man warn|These instructions, the make.py script etc.|are designed to operate in a Linux environment.  {{man menu|They won't work on Windows without modifications.}}}}
 
* Gramps uses Python version 3.2 or higher.  You must have at least that version installed.  If you have installed Gramps 4.2 or higher on your Linux system already, then a sufficient version of Python will be present. If you have more than one version of Python installed, then you must use the correct version for these scripts.  On some systems, both Python 2.x and 3.x are installed.  It is possible that the normal invocation of <code>python</code> starts up Python 2.x, and that to start up Python 3.x requires invoking with <code>python3</code> or <code>python3.4</code> etc.  You can test the version by <code>python –version</code> or <code>python3 –version</code>.  If this is so, replace any usage of 'python' in the examples below with the appropriate invocation.
 
* Gramps uses Python version 3.2 or higher.  You must have at least that version installed.  If you have installed Gramps 4.2 or higher on your Linux system already, then a sufficient version of Python will be present. If you have more than one version of Python installed, then you must use the correct version for these scripts.  On some systems, both Python 2.x and 3.x are installed.  It is possible that the normal invocation of <code>python</code> starts up Python 2.x, and that to start up Python 3.x requires invoking with <code>python3</code> or <code>python3.4</code> etc.  You can test the version by <code>python –version</code> or <code>python3 –version</code>.  If this is so, replace any usage of 'python' in the examples below with the appropriate invocation.
 
* The make.py used in construction of the addons requires that the LANGUAGE environment variable be set to 'en_US.UTF-8'.  
 
* The make.py used in construction of the addons requires that the LANGUAGE environment variable be set to 'en_US.UTF-8'.  
Line 110: Line 111:
  
 
=== Test your addon as you develop ===
 
=== Test your addon as you develop ===
 +
 +
{{man warn|{{bug|10436}} Symlinks to folders in gramps plugin dir are not scanned}}
 +
 
To test your addon as you develop it is suggested that you insert your NewProjectName plugin into your Gramps user plugin directory with a link to your addon development directory, like so:
 
To test your addon as you develop it is suggested that you insert your NewProjectName plugin into your Gramps user plugin directory with a link to your addon development directory, like so:
  
Line 117: Line 121:
 
Your installed Gramps will search this folder (and subdirectories) for .gpr.py files, and add them to the plugin list.
 
Your installed Gramps will search this folder (and subdirectories) for .gpr.py files, and add them to the plugin list.
  
If you have code that you want to share between addons, you don't need to do anything special. Currently, Gramps adds each directory in which a .gpr.py is found onto the PYTHONPATH which is searched when you perform an import. Thus "import NewProjectName" will work from another addon. You should always make sure you name your addons with a name appropriate for Python imports.
+
If you have code that you want to share between addons, you don't need to do anything special. Gramps adds each directory in which a .gpr.py is found onto the PYTHONPATH which is searched when you perform an import. Thus "import NewProjectName" will work from another addon. You should always make sure you name your addons with a name appropriate for Python imports.
  
 
=== Commit your changes ===
 
=== Commit your changes ===
Line 190: Line 194:
  
 
For general help on translations in Gramps, see [[Coding for translation]]. However, that will only use translations that come with Gramps, or allows you to contribute translations to the Gramps core. To have your own managed translations that will be packaged with your addon, read the rest of this page.
 
For general help on translations in Gramps, see [[Coding for translation]]. However, that will only use translations that come with Gramps, or allows you to contribute translations to the Gramps core. To have your own managed translations that will be packaged with your addon, read the rest of this page.
 +
Note that these instructions will only work for Python strings, if you have a glade file, it will not get translated.
  
 
For any addon which you have translations into other languages, you will need to add a way to retrieve the translation. You need to add this to the top of your NewProjectName.py file:
 
For any addon which you have translations into other languages, you will need to add a way to retrieve the translation. You need to add this to the top of your NewProjectName.py file:
Line 549: Line 554:
 
* https://github.com/gramps-project/addons-source  - Source code (Git)
 
* https://github.com/gramps-project/addons-source  - Source code (Git)
 
* https://github.com/gramps-project/addons - downloadable .tgz files
 
* https://github.com/gramps-project/addons - downloadable .tgz files
 
;Gramps Addons site for Gramps 3.2 to Gramps 4.1.
 
* https://sourceforge.net/p/gramps-addons/  - Source code (SVN) and downloadable .tgz files
 
  
 
[[Category:Developers/General]]
 
[[Category:Developers/General]]

Revision as of 04:35, 16 February 2018

Gnome-important.png
Warning:

This page documents the API, methods, and best practices for developing a Third-party Addon for Gramps 4.2 and later. For 4.1.x and earlier, see Addons development old.

Addons for Gramps can extend the program in many different ways. You can add any of the following types of addons:

  1. Report
  2. Quickreport
  3. Tool
  4. Importer
  5. Exporter
  6. Doc creator
  7. Plugin lib
  8. Map service
  9. Gramps View
  10. Relationships
  11. Gramplet
  12. Sidebar

Writing an addon is fairly straightforward if you have just a little bit of Python experience. And sharing your addon is the right thing to do. The general steps to writing an addon and sharing your own addons are:

  1. Develop your addon
  2. Create a Gramps Plugin Registration file (.gpr.py)
  3. Get translators to translate your addon into multiple languages
  4. Package your addon
  5. List and document your addon
  6. Support it through the issue tracker
  7. Maintain the code as Gramps continues to evolve

We'll now look at each of these steps in detail. Note that this wiki expects that addons are developed under Linux; while it is possible to do so under Windows, some of the steps will vary and the process has not been fully debugged, so user beware.

Develop your addon

The addons-source git repository has the following structure:

  • /addons-source
    • /IndividualNameOfAddon1
    • /IndividualNameOfAddon2
    • ...

The addons git repository has the following structure:

  • /addons
    • /gramps42
      • /download
      • /listings
    • /gramps50
      • /download
      • /listings

The addon-source repository holds the source code for the addons with branches holding the version for different gramps. If you are working on an addon for gramps for gramps42, be sure to use the gramps42 git branch, as the default is master branch (currently gramsp 5.x). Example commands are shown below.

Once you have your addon working in one branch, (say gramps42), then you will also want to copy it to the other branch (master), after you have tested it to work properly and edited the '.gpr.py' file appropriately.

Get a local copy of Gramps and its addons

These steps show how to download the addon sources.

  1. Get an https://github.com/join account if you don't already have one.
  2. Request GIT write access for the https://github.com/gramps-project/addons-source project by emailing the gramps-devel mailing list

See also git introduction for instructions on installing git and getting basic settings configured. Also https://help.github.com/articles/generating-an-ssh-key/ will help with setting up credentials for GitHub. To fully build and advertise a new addon will require local copies of the three repositories, the 'addons-source', 'addons' and the main Gramps source 'gramps'.

This wiki assumes that the three git repositories local locations are all to be put into the users' home directory and named with the repository names.

The three directories must be named as shown, and in the same base directory in order for the make.py script to work properly, however they don't have to be located directory in the users home directory.

From your home directory;

git clone [email protected]:gramps-project/addons-source.git addons-source
git clone [email protected]:gramps-project/addons.git addons
git clone [email protected]:gramps-project/gramps.git gramps

or if you do not have a Github account:

git clone https://github.com/gramps-project/addons-source.git addons-source
git clone https://github.com/gramps-project/addons.git addons
git clone https://github.com/gramps-project/gramps.git gramps

This will create a copy of the addons-source tree in your home directory at "~/addons-source", and the other trees at their respective locations as well.

To switch to a local copy of the gramps42 branch:

cd addons-source
git checkout -b gramps42 origin/maintenance/gramps42

or to work in the master branch:

cd addons-source
git checkout -b gramps42 origin/master

Other pre-requisites

Gnome-important.png
These instructions, the make.py script etc.

are designed to operate in a Linux environment. They won't work on Windows without modifications.

  • Gramps uses Python version 3.2 or higher. You must have at least that version installed. If you have installed Gramps 4.2 or higher on your Linux system already, then a sufficient version of Python will be present. If you have more than one version of Python installed, then you must use the correct version for these scripts. On some systems, both Python 2.x and 3.x are installed. It is possible that the normal invocation of python starts up Python 2.x, and that to start up Python 3.x requires invoking with python3 or python3.4 etc. You can test the version by python –version or python3 –version. If this is so, replace any usage of 'python' in the examples below with the appropriate invocation.
  • The make.py used in construction of the addons requires that the LANGUAGE environment variable be set to 'en_US.UTF-8'.
  • The make.py used in construction of the addons requires that the GRAMPSPATH environment variable be set to your path to the Gramps source tree.
  • intltool must be installed;
sudo apt-get install intltool

For example if your home directory is '/home/name' and you use the suggested path names, use

GRAMPSPATH=/home/name/gramps LANGUAGE='en_US.UTF-8' python3 make.py ...

to replace the ./make.py in the examples below.

Create your addon subdirectory

  • Make a new project directory in addons-source:
mkdir NewProjectName

Follow the development API for your tool

Create your NewProjectName.py and NewProjectName.gpr.py files.

Follow the development API for your tool, report, view, or Gramplets. Place all of your associated .py, .glade, etc. files in this directory. For general information on Gramps development see Portal:Developers and Writing a Plugin specifically.

Test your addon as you develop

Gnome-important.png
10436 Symlinks to folders in gramps plugin dir are not scanned

To test your addon as you develop it is suggested that you insert your NewProjectName plugin into your Gramps user plugin directory with a link to your addon development directory, like so:

cd ~/.gramps/gramps42/plugins
ln -s ~/addons-source/NewProjectName NewProjectName

Your installed Gramps will search this folder (and subdirectories) for .gpr.py files, and add them to the plugin list.

If you have code that you want to share between addons, you don't need to do anything special. Gramps adds each directory in which a .gpr.py is found onto the PYTHONPATH which is searched when you perform an import. Thus "import NewProjectName" will work from another addon. You should always make sure you name your addons with a name appropriate for Python imports.

Commit your changes

To commit your changes so that others can see your addon source.

  • Remove the files that should not be added to GitHub:
./make.py gramps42 clean NewProjectName
  • Add the project to the repository:
git add NewProjectName
  • Commit it with an appropriate message
git commit -m "A message describing what this addon is"

Before committing additional edits to your addon, you should:

  • to make sure that outside changes do not affect your commit
git pull --rebase
  • only the files you changed should be in this list
git status
  • Commit it with an appropriate message
git commit -m "A message describing the changes"

If you have been given 'push' rights to GitHub 'gramps-project/addons-source', and when you are sure you are done and want to publish to the repository:

  • to make sure that outside changes do not affect your commit
git pull --rebase
git push origin gramps42

Also you may want to Package your addon so it can be downloaded via the plugin manager.

Config

Some addons may want to have persistent data (data settings that remain between sessions). You can handle this yourself, or you can use Gramps' built-in configure system.

At the top of the source file of your addon, you would do this:

from config import config as configman
config = configman.register_manager("grampletname")
# register the values to save:
config.register("section.option-name1", value1)
config.register("section.option-name2", value2)
...
# load an existing file, if one:
config.load()
# save it, it case it didn't exist:
config.save()

This will create the file "grampletname.ini" and put in the same directory as the addon. If the config file already exists, it remains intact.

In the addon, you can then:

x = config.get("section.option-name1")
config.set("section.option-name1", 3)

and when this code is exiting, you might want to save the config. In a Gramplet that would be:

def on_save(self):
    config.save()

If your code is a system-level file, then you might want to save the config in the Gramps system folder:

config = configman.register_manager("system", use_config_path=True)

This, however, would be rare; most .ini files would go into the plugins directory.

In other code that might use this config file, you would do this:

from config import config as configman
config = configman.get_manager("grampletname")
x = config.get("section.option-name1")

Localization

For general help on translations in Gramps, see Coding for translation. However, that will only use translations that come with Gramps, or allows you to contribute translations to the Gramps core. To have your own managed translations that will be packaged with your addon, read the rest of this page. Note that these instructions will only work for Python strings, if you have a glade file, it will not get translated.

For any addon which you have translations into other languages, you will need to add a way to retrieve the translation. You need to add this to the top of your NewProjectName.py file:

from gramps.gen.const import GRAMPS_LOCALE as glocale
 _ = glocale.get_addon_translator(__file__).gettext


Then you can use the standard "_()" function to translate phrases in your addon.

You can use one of a few different types of translation functions:

  1. gettext
  2. lgettext
  3. ngettext
  4. lngettext
  5. sgettext

These have become obsolete in Gramps 4; gettext, ngettext, and sgettext always return translated strings in unicode for consistent portability between Python 2 and Python3.

See the python documentation for documentation of gettext and ngettext. The "l" versions return the string encoded according to the currently set locale; the "u" versions return unicode strings in Python2 and are not available in Python 3.

sgettext is a Gramps extension that filters out clarifying comments for translators, such as

_("Remaining names | rest")

Where "rest" is the English string that we want to present and "Remaining names" is a hint for translators.

Create a Gramps Plugin Registration file

First, create the NewProjectName.gpr.py file. The registration takes this general form:

register(PTYPE,
     gramps_target_version = "4.2",
     version = "1.0.0",
     ATTR = value,
)

PTYPE is TOOL, GRAMPLET, REPORT, QUICKVIEW, IMPORT, EXPORT, DOCGEN, GENERAL, MAPSERVICE, VIEW, or RELCALC.


ATTR depends on the PTYPE. But you must have gramps_target_version and version. gramps_target_version should be a string of the form "X.Y" version number matching Gramps X major, Y minor integer. version is a string of the form "X.Y.Z" representing the version of your addon. X, Y, and Z should all be integers.

Here is a sample Tool GPR file:

register(TOOL, 
         id    = 'AttachSource',
         name  = _("Attach Source"),
         description =  _("Attaches a shared source to multiple objects."),
         version = '1.0.0',
         gramps_target_version = '4.2',
         status = STABLE,
         fname = 'AttachSourceTool.py',
         authors = ["Douglas S. Blank"],
         authors_email = ["[email protected]"],
         category = TOOL_DBPROC,
         toolclass = 'AttachSourceWindow',
         optionclass = 'AttachSourceOptions',
         tool_modes = [TOOL_MODE_GUI]
         )

You can see examples of the kinds of addons here (for example, see gramps/plugins/drawreport/drawplugins.gpr.py) and see the full documentation here in the comments and docstrings.

Note that this .gpr.py will automatically use translations if you have them (see below). That is, the function "_" is predefined to use your locale translations; you only need to mark the text with _("TEXT") and include a translation of "TEXT" in your translation file. For example, in the above example, _("Attach Source") is marked for translation. If you have developed and packaged your addon with translation support, then that phrase will be converted into the user's language.

Report plugins

The possible report categories are (gen/plug/_pluginreg.py):

#possible report categories
CATEGORY_TEXT       = 0
CATEGORY_DRAW       = 1
CATEGORY_CODE       = 2
CATEGORY_WEB        = 3
CATEGORY_BOOK       = 4
CATEGORY_GRAPHVIZ   = 5
REPORT_CAT          = [ CATEGORY_TEXT, CATEGORY_DRAW, CATEGORY_CODE,
                        CATEGORY_WEB, CATEGORY_BOOK, CATEGORY_GRAPHVIZ]

Each report category has a set of standards and interface. The categories CATEGORY_TEXT and CATEGORY_DRAW use the Document interface of Gramps. See also Report API for a draft view on this.

The application programming interface or API for reports is treated at Report-writing_tutorial. For general information on Gramps development see Portal:Developers and Writing a plugin specifically.

General plugins

The plugin framework also allows you to create generic plugins for use. This includes the ability to create libraries of functions, and plugins of your own design.

Example: A library of functions

In this example, a file name library.py will be imported at time of registration (when Gramps starts):

# file: library.gpr.py

register(GENERAL, 
   id    = 'My Library',
   name  = _("My Library"),
   description =  _("Provides a library for doing something."),
   version = '1.0',
   gramps_target_version = '4.2',
   status = STABLE,
   fname = 'library.py',
   load_on_reg = True,
  )

The code in the file library.py will be imported when Gramps begins. You can access the loaded module in other code by issuing an "import library" as Python keeps track of files already imported. However, the amount of useful code that you can run when the program is imported is limited. You might like to have the code do something that requires a dbstate or uistate object, and neither of these is available when just importing a file.

If "load_on_reg" was not True, then this code would be unavailable until manually loaded. There is no automatic mechanism in Gramps to load GENERAL plugins automatically.

In addition to importing a file at startup, one can also run a single function inside a GENERAL plugin, and it will be passed the dbstate, the uistate, and the plugin data. The function must be called "load_on_reg", and take those three parameters, like this:

# file: library.py

def load_on_reg(dbstate, uistate, plugin):
    """
    Runs when plugin is registered.
    """
    print("Hello World!")

Here, you could connect signals to the dbstate, open windows, etc.

Another example of what one can do with the plugin interface is to create a general purpose plugin framework for use by other plugins. Here is the basis for a plugin system that:

  • allows plugins to list data files
  • allows the plugin to process all of the data files

First, the gpr.py file:


register(GENERAL, 
  id    = "ID",
  category = "CATEGORY",
  load_on_reg = True,
  process = "FUNCTION_NAME",
  )

This example uses three new features:

  1. GENERAL plugins can have a category
  2. GENERAL plugins can have a load_on_reg function that returns data
  3. GENERAL plugins can have a function (called "process") which will process the data

If you (or someone else) create additional general plugins of this category, and they follow your load_on_reg data format API, then they could be used just like your original data. For example, here is an additional general plugin in the 'WEBSTUFF' category:

# anew.gpr.py

register(GENERAL, 
  id    = 'a new plugin',
  category = "WEBSTUFF",
  version = '1.0',
  gramps_target_version = '5.0',
  data = ["a", "b", "c"],
  )

This doesn't have load_on_reg = True, nor does it have a fname or process, but it does set the data directly in the .gpr.py file. Then we have the following results:

>>> from gui.pluginmanager import GuiPluginManager
>>> PLUGMAN = GuiPluginManager.get_instance()
>>> PLUGMAN.get_plugin_data('WEBSTUFF')
["a", "b", "c", "Stylesheet.css", "Another.css"]
>>> PLUGMAN.process_plugin_data('WEBSTUFF')
["A", "B", "C", "STYLESHEET.CSS", "ANOTHER.CSS"]

Registered GENERAL Categories

The following are the published secondary plugins API's (type GENERAL, with the following categories):

WEBSTUFF

A sample gpr.py file:

# stylesheet.gpr.py

register(GENERAL, 
  id    = 'system stylesheets',
  category = "WEBSTUFF",
  name  = _("CSS Stylesheets"),
  description =  _("Provides a collection of stylesheets for the web"),
  version = '1.0',
  gramps_target_version = '5.0',
  fname = "stylesheet.py",
  load_on_reg = True,
  process = "process_list",
  )

Here is the associated program:

# file: stylesheet.py

def load_on_reg(dbstate, uistate, plugin):
    """
    Runs when plugin is registered.
    """
    return ["Stylesheet.css", "Another.css"]

def process_list(files):
    return [file.upper() for file in files]

Filters

For example:

register(GENERAL,
   category="Filters",
   ...
   load_on_reg = True
)
def load_on_reg(dbstate, uistate, plugin):
    # returns a function that takes a namespace, 'Person', 'Family', etc.

    def filters(namespace):
        print("Ok...", plugin.category, namespace, uistate)
        # return a Filter object here
    return filters

Get translators to translate your addon into multiple languages

  • Initialize and update the template.pot for your addon:
cd ~/addons-source
./make.py gramps42 init NewProjectName
  • You should edit the header of template.pot with your information, so it gets copied to individual language files.
  • Initialize a language for your addon (say French, fr):
./make.py gramps42 init NewProjectName fr
  • Update it from gramps and other addons:
./make.py gramps42 update NewProjectName fr
  • Edit the translations file manually:
/NewProjectName/po/fr-local.po
  • Compile the language:
./make.py gramps42 compile NewProjectName
  • Add or update your local language file, and commit changes:
git add NewProjectName/po/fr-local.po
git commit NewProjectName/po/fr-local.po -m "Added fr po file"
  • If you have been given 'push' rights to GitHub 'gramps-project/addons-source', then;
git push origin gramps42

Package your addon

To create a downloadable package:

./make.py gramps42 build NewProjectName or
./make.py gramps50 build NewProjectName for the master branch.

Note that the

This will automatically include the following files in your build:

  • *.py
  • *.glade
  • *.xml
  • *.txt
  • locale/*/LC_MESSAGES/*.mo

Starting with Gramp 5.0, if you have additional files beyond those listed above, you should create a MANIFEST file in the root of your addon folder listing the files (or pattern) one per line, like this sample MANIFEST file:

README.md
extra_dir/*
help_files/docs/help.html
Gramps-notes.png
Note:

Running the command make.py xxx build will increment the third number in your dotted version number of all addons in the gpr.py file. Consider this number to be a "build number".

This will leave your 'addons-source' with untracked changes according to git. You should delete the 'NewProjectName/locale' directory. The updated 'NewProjectName/NewProjectName.gpr.py ' is ready to add and commit the next time you make other changes.

rm –rf –v 'NewProjectName/locale'

Then add the package to GitHub:

 cd '~/addons'
 git add gramps42/download/NewProjectName.addon.tgz
 git commit -m "Added new plugin: NewProjectName"

or (for the master branch);

 cd '~/addons'
 git add gramps50/download/NewProjectName.addon.tgz
 git commit -m " Added new plugin: NewProjectName"


List your addon in the Gramps Plugin Manager

Gnome-important.png
Gramps needs to have been built

Make sure you have already built gramps42 or master. Change to the appropriate git branch in your gramps directory, and run 'python3 setup.py build' See Linux:Build_from_source

To create a listing:

cd '~/gramps-addons' or wherever you have built your addon
GRAMPSPATH=path/to/your/gramps/install ./make.py gramps42 listing NewProjectName

or (for the master branch);

cd '~/gramps-addons' or wherever you have built your addon
GRAMPSPATH=path/to/your/gramps/install ./make.py gramps50 listing NewProjectName

That will create a series of files in the ../listings/ directory.

Then add the updated listing to GitHub:

 cd '~/addons'
 git add gramps42/listings/*
 git commit -m "Added new plugin to listings: NewProjectName"

or (for the master branch);

 cd '~/addons'
 git add gramps50/listings/*
 git commit -m " Added new plugin to listings: NewProjectName"

List and document your addon on the wiki

Document the addon in the wiki using the name Addon:NewProjectName.

Edit 4.2_Addons or 5.0_Addons and describe your addon.

You can point to the addon.tgz in GitHub as the downloadable file.

Miscellaneous commands

To build and compile translations for all projects to their download/Addon.addon.tgz files:

python3 make.py gramps42 build all

To compile translations for all projects :

python3 make.py gramps42 compile all

Support it through issue tracker

Visit https://gramps-project.org/bugs/view_all_bug_page.php and become a user. Suggest to check it regularly.

Maintain the code as Gramps continues to evolve

Remember that Gramps addons exist for many reasons and there are many Gramps developers that do support addons in various ways (translations, triage, keeping in sync with master, download infrastructure, etc).

Some reasons why the addons exist; they provide:

  • A quick way for anyone to share their work; the Gramps-project has never denied adding a addon.
  • A method to continuously update and develop a stand-alone component, often before being officially accepted.
  • A place for controversial plugins that will never be accepted into core, but are loved by many users (eg, Data Entry Gramplet).
  • A place for experimental components to live.

Resources

Gramps Addons site for Gramps 4.2 and newer