Proposition of Many Gramps users would like to collaborate or share their genealogy data on the web. This GEP describes a webapp, a web-based application that runs in your browser, and requires a server mode for GRAMPS. Now that A prototype is on-line at http://gramps-connect.org/ which is running trunk on a sample database. You can log into the main graphical site, as a:*superuser (id=admin, password=gramps) or a *regular user interface (GUIid=admin1, password=gramps) has been separated from the commandor just view as an anonymous user. There are two additional pages on this project: * [[Gramps-Connect]] - getting started* [[Gramps-line interface (CLI), a serverConnect: Developer Introduction]] -mode would be the next logical step.introduction for developers
== Motivation ==
Web developers are in need The main focus of a method of accessing and creating functionality with Gramps-based webapp is collaboration. The Gramps webapp will allow users to easily move their GRAMPS genealogy data on to the webto be seen, and edited with proper login and permissions, in a live, collaborative environment. Having Here is a GRAMPS serversmall list of goals: # Create a fullscale Gramps web framework # Allow multiple users via the standard web browser## Users will log in and have various levels of permissions# Build on Gramps codebase and wealth of resources## Reports## Tools## Visualizations## Date and calendar functions## Translations## Manual and documentation# Use standards and well-mode would allow a GRAMPSknown, well-tested frameworks where possible## WSGI protocol for running code## Django framework## Underlying powerful database engines === FAQ === 1. ''Aren't there already many fine, web-based web project.genealogy programs? Why don't you just use one of those? Aren't you re-inventing the wheel?''
== Sample Usage ==There are indeed many fine, web-based genealogy programs, and some are even free software/open source. However, there are a few good reasons to develop a Gramps-based webapp:
A server mode for GRAMPS is suggested to work as follows# Gramps has hundreds of thousands of lines of code, some of which could be re-used directly in a webapp. A server program is startedFor example, the reports could be run and run in downloaded directly from the backgroundwebapp. It listens # Gramps has a very well-defined set of tables and relationships that could be re-implemented for requests from other processes (on -line use. # Users have grown to appreciate the design of Gramps, and we want to continue to build on this design.# Many users want to collaborate. Currently, they would either have to move their data in and out of Gramps, or give up Gramps completely.# We want to keep the developers and users that machinewe have, or others)and so not splinter our groups. By building the webapp on top of core gramps code, executes commandswe continue to refine and make better our current code, and serves back datakeep our current developers working on the parts that they know and love.
The server mode is vary similar to the CLI mode of GRAMPS2. However, rather than just running ''Why do you need a report, the program doesnweb framework like Django? Can't exityou just use the same Python code, but waits for requests. Clients need not keep track of any state. The server would host a single and same database, and allow multiple people to work on it simultaneously.that Gramps already uses?''
== Functionality ==We can't use the same database (what is called a "backend") directly. Currently Gramps uses BSDDB, and it is not configured for use in a multiuser, client/server environment. But even if we could use the same backend, we would still want some type of web development framework. Django is one of the best in any language, and it just happens to be in Python.
A GRAMPS server should allow clients 3. ''How easy will this be for me to do most of what a user would want to do with the GRAMPS gtk application:use on my website?''
# browse dataWe have designed it to be as easy as it can be, given that we are using Python. Many web sites allow Python programs, including all of and Django allows many different variations in running. We picked the primary objects# have links that would take protocol with the user directly to view a particular objectmost availability (WSGI). Don't worry if you haven's data# a method t heard of adding and editing existing datait. Your webserver can probably run it.
== Communication ==4. ''When is this going to be available?''
In order for such a client/server architecture We are hoping to work, there needs to be have a method of communication between them. There needs to be a format fully functioning webapp ready for making requests from clients to the server, and a for returning results from the server to the clientstesting July 2010.
There are plenty of options for communication formats, including XML5. There is also a library called [http://pyro.sourceforge.net/ Python Remote Objects] for handing this kind of data. However, one possibility is to use Python's native pickle format, as that 'How can be easily sent between connections. Likewise, all of the primary dataI help?''s raw data can be pickled. Using pickled data requires that no database objects be pickled---only data. One would have to be careful about how you write web applications so as not to involve objects such as generators, and databases.
== Prototype ==You can start by reading the rest of this page and sending ideas and comments to the Gramps-developers mailing list, and running the code if you can.
A prototype of a GRAMPS server mode has been checked into gramps/trunk (targeting version 3.2). It works as follows.== Overview ==
You start The Gramps webapp is written in Django. Django is a sophisticated, modern web development framework. Django is written in Python, albeit in a very different style from Gramps. However, part of the server by selecting motivation of using Django is that it breaks up web development into very clearly defined parts following the database[http://en.wikipedia.org/wiki/Model_view_controller Model-View-Controller] paradigm. Two of these parts require no special programming knowledge, and thus will allow more people to be able to possibly customize and participate in the Gramps project.
python src/gramps.py The Gramps webapp (and Django in general) is broken into three well--server=50000 -O "My Family Tree"defined parts:
This will start a socket listener on port 50000. If the database is locked, then, like usually, you can supply the --force-unlock flag:# models/views# templates # CSS (Cascading Style Sheets)
python srcThe models define the tables and relationships, but this is done in Python (not SQL). The models also define the API to read/grampswriting/editing the data. The views are also written in Python, and are closely tied to the models.py The templates are written in HTML and a template language that is very easy for non-programmers to use and understand. Finally, CSS is just Cascading Style Sheets, where all of the graphical definitions are made. The webapp uses pre-server=50000 -O existing CSS created for the "My Family TreeNarrated Web" --force-unlockreport of Gramps which was used for created static web pages. Let's take a look at specific examples of each of these parts.
In fact, you can do any of the things you normally do with the CLI, before beginning serving.=== Models/Views ===
After starting, Here is the server produces output likemodel that defines the Person table from [{{Code Base}}gramps/webapp/grampsdb/models.py gramps/webapp/grampsdb/models.py]:
<pre>
$ python src/gramps.py --serverclass Person(PrimaryObject): gender_type =50000 -O "MyRelate"Opened successfully!GRAMPS server listening on port 50000models.ForeignKey('GenderType')Use CONTROL+C to exit.. families = models.ManyToManyField('Family', blank=True, null=True)--------------------------------------------------Connection opened from 127 parent_families = models.0.0.1:50114ManyToManyField('Family', Connection closed from 127.0.0.1:50114 related_name="parent_families",Connection opened from 127.0.0.1:50115 blank=True, null=True) Request: self.dbstate.db.surname_listreferences = generic.__getslice__GenericRelation(0'PersonRef', related_name="refs", 2147483647 content_type_field="object_type", object_id_field="object_id")Connection closed from 127.0.0.1:50115
</pre>
The server currently lists each connection start/endHere, and each request. The server is written in a fashion such that it you can take many requests at once. It spawns off threads see that handle the details of the request. TODOPerson only has 4 parts: does memory get reclaimed appropriately? The server receives messages that it evaluatesgender_type, families, picklesparent_families, and returnsreferences. To make programming as natural as possible on There are additional properties, but they are defined in the client side, a RemoteObject has been written PrimaryObject class which hides the gory details of the socket communicationis shared with other tables. Here is a sample clientPrimaryObject:
<pre>
from cliclass PrimaryObject(models.Model): class Meta: abstract = True id = models.AutoField(primary_key=True) handle = models.CharField(max_length=19, unique=True) gramps_id = models.CharField('gramps id', max_length=25, blank=True) last_saved = models.client import RemoteObjectDateTimeField('last changed', auto_now=True) self last_changed = RemoteObjectmodels.DateTimeField("localhost"'last changed', 50000null=True, blank=True) # user edits private = models.BooleanField('private') marker_type = models.ForeignKey('MarkerType')
</pre>
HereThe big difference here between typical Python programming is that the Person class defines the Person table, you can use "self" very similarly and the interface to the use it. Most Python code would probably have Person be an instance of self in a class, but Django uses classes to represent many things. Here are three examples using the server. For examplePerson class:
<pre>
$ python % cd trunkPython 2.6 (r26:66714, Jun 8 2009, 16:07:26) [GCC 4.4.0 20090506 (Red Hat 4.4.0-4)] on linux2Type "help", "copyright", "credits" or "license" for more information % PYTHONPATH=src DJANGO_SETTINGS_MODEL=webapp.settings python >>> from cliwebapp.grampsdb.client models import RemoteObjectPerson >>> self = RemoteObjectPerson.objects.all("localhost") [<Person>, <Person>, 50000)...] >>> selfPerson.dbstateobjects.get(id=1) <DbState.DbState object at 0x98e088cPerson> >>> selfPerson.dbstateobjects.dbget(handle='gh71234dhf3746347734') <gen.db.dbdir.GrampsDBDir object at 0x9dc4c0cPerson>
</pre>
Under The first retrieves all of the hoodrows for the Person table; the second retrieves just the one record that has the unique, primary key 1, the names "db" and "dbstate" are collected, sent over the socket to third retrieves the server as single record that has the string "selfunique handle of 'gh71234dhf3746347734'. Note that we never connected onto a database..dbstate.db" which Django is evaluated(currently) define to connect on to one database, and the representation it does it on import. The database is sent backset in gramps/webapp/settings. Notice that the server cannot return an entire dbstate or db object to the client, but it merely returns the repr stringpy.
You can An alternative method of interactively explore talking to the remote objects, toodatabase is to use ''manage.py'':
<pre>
>>> self.dbstate % cd master/gramps/webapp % PYTHONPATH=.db.dir()['_Callback__BLOCK_ALL_SIGNALS', '_Callback__LOG_ALL', '_Callback__block_instance_signals',/../gramps python manage.py shell'source_prefix', 'surname_list', 'surnames', 'transaction_begin', 'transaction_commit', 'translist', 'txn', 'undo', 'undo_available', 'undo_callback', 'undo_data', 'undo_history_callback', 'undo_history_timestamp', 'undo_reference', 'undodb', 'undoindex', 'undolog', 'update_empty', 'update_real', 'update_reference_map', 'url_types', 'version_supported', 'write_version'] >>>
</pre>
Notice that That will give you can't wrap a "dir()" around a propertyan ipython shell, but if you can tack a ".dir()" on the end to provide the same functionality. You can't wrap a dir() around the self.dbstate.db because that would get applied on the client side, and by the time self.dbstate.db is evaluated, have it is just the repr stringinstalled.Very nice environment!
But, you You can also get back some full GRAMPS objectsuse the Person interface to select a subset of people:
<pre>
>>> selffrom webapp.dbstategrampsdb.dbmodels import * >>> Person.get_default_personobjects.filter(gender_type=1) [<genPerson>, <Person>, .lib.person.Person object at 0xb7d7ac6c>]
</pre>
This isn't just the repr string returned this time, but a real object. You can test that byor even more clearly:
<pre>
>>> p = selfPerson.dbstateobjects.db.get_default_personfilter(gender_type__name="Male") [<Person>, <Person>> p, ..get_primary_name().get_surname()u'Blank']
</pre>
Because you canThe double-underscore in the keyword "gender_type__name" of the filter method is a Django convention. It means "replace with the correct syntax". If Python allowed it, it would be written as ''t get back generators nor database 'Person.objects, sometimes you need to do some processing on .filter(gender_type.name="Male")''' but that is not legal syntax. ==== Model overview ====Here is an overview of all of the servermodels and how they are related: [[Image:all-tables. gif]] {{out of date}}{{man tip| 1=To do update this(Gramps 3.x and earlier) |2=To see more graphical representations of the data, run "make docs" in the src/webapp/ directory, you can use selfand then look in src/webapp/docs/.remote:}}
>>> self* [https://gramps-project.remote("person = selforg/docs/gen/gen_db.sdb.name(self.dbstate.db.get_default_person())")html#dbdjango Gramps DbDjango]
self=== Templates === Templates are used to describe ''what'' to display.remote takes Here is a string and sends it to the server to be evaluated (or executed)template from [{{Code Base}}data/templates/main_page.html data/templates/main_page. self containshtml]:
<pre>
{% extends "gramps-base.html" %} {% block title %}GRAMPS Connect - main page {% endblock %}{% block heading %}GRAMPS - main page {% endblock %} {% block content %} <p id="description">Welcome to GRAMPS Connect, a new web-based collaboration tool. {% if user.is_authenticated %} You are now logged in as <a href="/user/{{user.username}}">{{user.username}}</a> self.dir()['__doc__', '__init__', '__module__', 'arghandler', 'climanager', 'dbstate', 'env', 'eval', 'reset', 'sdb']{% endif %}</p> <p id="description">Database information:<ul>{% for view in views %} <li><a href="/{{view|lower}}">{{view}}</a></li>{% endfor %}</ul></p>{% endblock %}
</pre>
* arghandler === CSS === Finally, here is a screen shot of the main_page.html (above) showing some initial testing of Gramps in Django using the Mainz CSS from the NarrWeb report: [[Image:Gramps_in_django.gif]] === Getting Started with Gramps in Django === A prototype of a Gramps Django webapp is now in trunk and gramps32. To run it, do the following: # Download Django version 1.3 or greater## On yum-based systems, try "yum install Django"## On apt-based systems, try "sudo apt-get install python-django"## Other systems: get the sources from http://www.djangoproject.com/download/# clone the Git repository and checkout either the gramps32 or master branch # cd src/web/# Build the database, and load with default data:## make clean## make## This will ask for an id, email, and password for a superuser. You can add one later if you don't do it now.# Run the test webserver:## make run# Point your webbrowser to:## http://127.0.0.1:8000/ At this point, you can now export your Gramps data to Django (and back). In another terminal window: # Start up gramps:## cd ../..## python src/gramps.py# Download the Django Import/Export Addon from [[3.3_Addons]]# Run the Django Exporter## Select Family Tree -> Export## Select Django This will export your regular Gramps BSDDB data into whatever Django database you have defined in settings.py above. You now have your data in a sqlite SQL database, and can access it via the webbrowser. To import data back from Django's SQL table back into Gramps from the website: # Create a file named "import.django" somewhere (just needs to end in ".django").# Start up this version of Gramps## python src/gramps.py# Run the Django Importer## Select Family Tree - > Import [[Image:DjangoImportExport.jpg|thumb|right|150px]]## Select the "import.django" (from above) as the object that handles file to import To add a superuser (after the CLI operationsinitialization): # cd src/web# PYTHONPATH=../../src python manage.py createsuperuser For more on Django, try their tutorial: * climanager Tutorial: http://docs.djangoproject.com/en/dev/intro/tutorial01/#intro- tutorial01 === Webapp Files === There are two subdirectories and two files of interest to the Gramps webapp:* dbstate # {{Code Base}}data/templates/ - HTML templates# {{Code Base}}gramps/webapp/ - Webapp main directory## {{Code Base}}gramps/webapp/libdjango.py - library interface## {{Code Base}}gramps/webapp/grampsdb - gramps table models# http://gramps-addons.svn.sourceforge.net/viewvc/gramps-addons/trunk/contrib/Django/ExportDjango.py?view=markup - Exporter# http://gramps-addons.svn.sourceforge.net/viewvc/gramps-addons/trunk/contrib/Django/ImportDjango.py?view=markup - holds Importer == Roadmap == Phase 1: get the basic Django skeleton in place, including the core HTML templates, models, views, and templatetags. Should be able to browse the 8 primary tables. Get translations in place. Goal for version 0.1 to be announced with Gramps 3.2 in March 2010. Phase 2: Be able to run all of the reports directly from the web with an option interface. Be able to import/export from the web. This will largely depend on a gen/db/dbdjango library. Goal for version 0.5beta, May 2010. Phase 3: add and edit data from the web. This would complete the state functionality of the databaseweb interface. Goal July 2010. Phase 4: Refine and polish. Release with Gramps 3.3. If you would like to work on an area, please note it here: # Kathy - edits and adding new data# Doug - Integration with gramps core; browsing data# - Translation system# - Proxy interface to show Private data# - concurrent edits# - date widget# - running reports interface* dbstate# - media files...db where do they go?# - options interface, for editing options to run report# - import GEDCOM from web# - the databasefull djangodb.py to replicate all functions of bsddb* sdb # - simple database user support (email, mailing lists, permissions, etc) == Issues == === Concurrent Edits === Concurrent accessfor write and read imply several problems when people by accident change the same objects at the same time. Gramps itself has an elaborate signal handling for cases when dialogs are open with no longer current information. In a web environment, this becomes more difficult however. This is not built into Django.
FinallyFor discussion on this issue in Django, here is an example for listing out the surnames of a family tree on the websee:
<pre>* [http://groups.google.com/group/django-users/browse_thread/thread/c138ec11c6ad282e?hl=en# Django User Question] #!** [http://groups.google.com/group/django-developers/usrbrowse_thread/binthread/pythonfd5d45fc6cd6a760 Developer discussion on topic]
import sys, ossys.path.append("/home/dblank/gramps/trunk/src")os.environ["HOME"] = "/home/dblank/html/gramps"= Example GMS Web Sites ==
from cli.client import *We now have a example gramps webapp on the web:
self = RemoteObject("localhost", 50000)* http://gramps-connect.org/
print "Content-typeGenealogy Management Systems on the web: text/html"printprint "<html>"print "<body>"print "First Demo of GRAMPS --server"
surnames = self* http://www.dertinger.de/Dertinger_database/en/en_index.htm (Oxy-gen)* http://www.admiraal.org (PhpGedView): Note here: the intro page is a collection of gadgets/controls, which then link into the real data.* http://webtrees.net/demo/next (webtrees)* http://beck.org.il/humogen/ (HuMogen)* http://genealogies.geneamania.net/servin/ (Généamania)* http://www.geneotree.com/geneotree/index.php (Geneotree)* http://ancestorsnow.com/ancestors* http://www.phpmyfamily.net/demo/* http://www.frog.dbstateza.dbnet/family/surname-list.surname_listphp ([[:Other_genealogy_tools#Gramps-php-exporter|gramps-php-exporter]])
for name in surnames: print "<li><a href=\"?surname=%s\">%s<Collaborative database (user/a><wizard/li>" % (name, namepassword):
print "<hr>"* http://roglo.eu/roglo?lang=en (GeneWeb)print "<* http://body>"print "<gennus.org ([http://beta.gennus.org/en/page/about.html beta][http://beta.gennus.org/en/page/releasenotes.html>"])<* http://brozer.fr (alpha[http://www.innovup.com/evenement/124/pre>89-actualites-agenda.htm])
and the resulting screen shotSource oriented:
* http://solumslekt.org/forays/yggdrasil.php [http://code.google.com/p/yggdrasil-genealogy/][Imagehttp:GRAMPS-server//solumslekt.png]org/blog/]
== Discussion See also== *[[Gramps-Connect: Introduction|gramps-connect]]
[[Category:GEPS|SG]]
