Gramps Server Mode

From Gramps
Revision as of 22:49, 31 January 2013 by Patsyblefebre (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search


This was part of GEPS 013: Gramps Webapp. However, this has been superseded by a larger effort to build an entire web application. These notes are left here in case they might be useful to someone. The prototype code for this functionality is in the branch/gep/geps-013-server SVN.

A server mode for GRAMPS is suggested to work as follows. A server program is started, and run in the background. It listens for requests from other processes (on that machine, or others), executes commands, and serves back data.

The server mode is very similar to the CLI mode of GRAMPS. However, rather than just running a report, the program doesn't exit, but waits for requests. Clients need not keep track of any state. The server would host a single database, and allow multiple people to work on it simultaneously.

Functionality

A GRAMPS server should allow clients to do most of what a user would want to do with the GRAMPS gtk application:

  1. browse data, including all of the primary objects
  2. have links that would take the user directly to view a particular object's data
  3. a method of adding and editing existing data

Communication

In order for such a client/server architecture to work, there needs to be a method of communication between them. There needs to be a format for making requests from clients to the server, and a for returning results from the server to the clients.

There are plenty of options for communication formats, including XML. There is also a library called Python Remote Objects for handing this kind of data. However, some possibilities:

  • use Python's native pickle format, as that can be easily sent between connections.
  • use a simple XML schema
  • use the GRAMPS XML schema

Whichever method is chosen, all of the primary data's raw data can be encoded. Using encoded data requires that no database objects be encoded---only data. One would have to be careful about how you write web applications so as not to involve objects such as generators, and databases.

Prototype

A prototype of a GRAMPS server mode has been checked into gramps/trunk (targeting version 3.2). You may have to run "./configure" and "make" after doing an "svn update". It works as follows.

Server

You start the server by selecting the database and listening port:

python src/gramps.py --server=50000 -O "My Family Tree"

(If you don't know your tree names, use "python src/gramps.py -L").

This will start a socket listener on port 50000. If the database is locked, then, like usually, you can supply the --force-unlock flag:

python src/gramps.py --server=50000 -O "My Family Tree" --force-unlock

In fact, you can do any of the things you normally do with the CLI, before beginning serving.

After starting, the server produces output like:

$ python src/gramps.py --server=50000 -O "MyRelate"
Opened successfully!
GRAMPS server listening on port 50000.
Use CONTROL+C to exit...
--------------------------------------------------
Connection opened from 127.0.0.1:50114
Connection closed from 127.0.0.1:50114
Connection opened from 127.0.0.1:50115
    Request: self.dbstate.db.surname_list.__getslice__(0, 2147483647)
Connection closed from 127.0.0.1:50115

The server currently lists each connection start/end, and each request. The server is written in a fashion such that it can take many requests at once. It spawns off threads that handle the details of the request. TODO: does memory get reclaimed appropriately?

The server receives messages that it evaluates, encodes it (pickle or XML), and returns.

Interactive Client

To make programming as natural as possible on the client side, a RemoteObject has been written which hides the gory details of the socket communication. Here is a sample client:

from cli.client import RemoteObject
self = RemoteObject("localhost", 50000)

Here, you can use "self" very similarly to the use of self in the server. For example:

$ python
Python 2.6 (r26:66714, Jun  8 2009, 16:07:26) 
[GCC 4.4.0 20090506 (Red Hat 4.4.0-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from cli.client import RemoteObject
>>> self = RemoteObject("localhost", 50000)
>>> self.dbstate
<DbState.DbState object at 0x98e088c>
>>> self.dbstate.db
<gen.db.dbdir.GrampsDBDir object at 0x9dc4c0c>

Under the hood, the names "db" and "dbstate" are collected, sent over the socket to the server as the string "self.dbstate.db" which is evaluated, and the representation is sent back. Notice that the server cannot return an entire dbstate or db object to the client, but it merely returns the repr string.

You can interactively explore the remote objects, too:

>>> self.dbstate.db.dir()
['_Callback__BLOCK_ALL_SIGNALS', '_Callback__LOG_ALL', '_Callback__block_instance_signals',
...
'source_prefix', 'surname_list', 'surnames', 'transaction_begin', 'transaction_commit', 
'translist', 'txn', 'undo', 'undo_available', 'undo_callback', 'undo_data', 
'undo_history_callback', 'undo_history_timestamp', 'undo_reference', 'undodb', 
'undoindex', 'undolog', 'update_empty', 'update_real', 'update_reference_map', 'url_types', 
'version_supported', 'write_version']

Notice that you can't wrap a "dir()" around a property, but you can tack a ".dir()" on the end to provide the same functionality. You can't wrap a dir() around the self.dbstate.db because that would get applied on the client side, and by the time self.dbstate.db is evaluated, it is just the repr string.

But, you can also get back some full GRAMPS objects:

>>> self.dbstate.db.get_default_person()
<gen.lib.person.Person object at 0xb7d7ac6c>

This isn't just the repr string returned this time, but a real object. You can test that by:

>>> p = self.dbstate.db.get_default_person()
>>> p.get_primary_name().get_surname()
u'Blank'

Because you can't get back generators nor database objects, sometimes you need to do some processing on the server. To do this, you can use self.remote:

>>> self.remote("name = self.sdb.name(self.dbstate.db.get_default_person())")
>>> self.remote("name")
u'Blank, Living'

self.remote takes a string and sends it to the server to be evaluated (or executed). self contains:

>>> self.dir()
['__doc__', '__init__', '__module__', 'arghandler', 'climanager', 'dbstate', 'env', 'eval', 'reset', 'sdb']
  • arghandler - the object that handles the command-line operations
    • dbman - Functions related to database management
      • active
      • break_lock
      • create_new_db_cli
      • current_names
      • dbstate
      • empty
      • family_tree_list
      • family_tree_summary
      • get_dbdir_summary
      • get_family_tree_path
      • icon_values
      • import_new_db
      • is_locked
      • msg
      • needs_recovery
  • climanager - low-level CLI management
    • db_loader
    • dbstate
    • do_load_plugins
    • file_loaded
    • open_activate
  • dbstate - holds the state of the database
    • dbstate.db - the database
  • sdb - simple database access

Web Client

Finally, here is an example for listing on the web the properties and surnames of a family tree:

#!/usr/bin/python

import sys, os
sys.path.append("/home/dblank/gramps/trunk/src")
os.environ["HOME"] = "/home/dblank/html/gramps"

from cli.client import *

self = RemoteObject("localhost", 50001)

print "Content-type: text/html"
print
print "<html>"
print "<body>"
print "<h1>First Demo of GRAMPS --server</h1>"

dbfile = self.dbstate.db.full_name
people, dbversion = self.arghandler.dbman.get_dbdir_summary(dbfile)
summary_list = self.arghandler.dbman.family_tree_summary()
summary_dict = {}

for sdict in summary_list:
    if sdict["Path"] == dbfile:
        summary_dict = sdict

print "<h2>Properties:</h2>"
for prop in summary_dict:
    print "<b>%s</b>: %s <br/>" % (prop, summary_dict[prop])

print "<h2>Surnames:</h2>"
surnames = self.dbstate.db.surname_list[:10]
for name in surnames:
    if len(name) == 0:
        name = "[Missing Surname]"
    print "<li><a href=\"?surname=%s\">%s</a></li>" % (name, name)

print "<hr>"
print "</body>"
print "</html>"

Save the above in a file ending with .cgi. I used index.cgi. I then configured apache to allow CGI, added index.cgi, turned off SELinux, and started up the httpd service.

The resulting screen shot:

Screenshot-server.gif

Notes

Sometimes you may need to add a new function on the server. The original db.find_backlink_handles returned a generator, but we can't use that. Here it is as a list:

$ python -i src/cli/client.py localhost 50002
GRAMPS Remote interface; use 'self' to access GRAMPS
>>> db = self.dbstate.db
>>> default_person = db.get_default_person()
>>> db.find_backlink_handles_list(default_person.handle)

Note that a remote reference (not a function call) such as self.dbstate.db.full_name returns a reference to a temporary object that will repeatedly contact the server when referenced.

>>> full_name = self.dbstate.db.full_name

No server access.

>>> full_name
'/home/dblank/.gramps/grampsdb/4a79400a'
>>> full_name
'/home/dblank/.gramps/grampsdb/4a79400a'

Accesses the server twice. To make these types of references actual values on the client side, use something like:

>>> full_name = self.dbstate.db.full_name[:]
>>> full_name
'/home/dblank/.gramps/grampsdb/4a79400a'
>>> full_name
'/home/dblank/.gramps/grampsdb/4a79400a'

Only one server access (the assignment). Afterwards it is local.