Difference between revisions of "Addon:Lxml Gramplet"
m (→CherryTree gramplet ?) |
|||
(39 intermediate revisions by 3 users not shown) | |||
Line 1: | Line 1: | ||
− | ''lxml gramplet'' is an experimental [[Gramplets|gramplet]] working under POSIX platform(s), which reads, writes, transforms our [[ | + | {{Third-party plugin}} |
+ | |||
+ | '''lxml gramplet''' is an experimental [[Gramplets|gramplet]] working under POSIX platform(s), which reads, writes (''not the original one; safe read only state''), transforms content of our [[Gramps XML]] file on the fly without an import into our database (Gramps session). | ||
==Dependencies and file format== | ==Dependencies and file format== | ||
* [http://lxml.de/ lxml] is a Pythonic binding for the C libraries [http://xmlsoft.org/ libxml2] and [http://xmlsoft.org/XSLT/ libxslt]. It is known for good performances by using C-level ([http://www.cython.org/ Cython]). | * [http://lxml.de/ lxml] is a Pythonic binding for the C libraries [http://xmlsoft.org/ libxml2] and [http://xmlsoft.org/XSLT/ libxslt]. It is known for good performances by using C-level ([http://www.cython.org/ Cython]). | ||
− | * [[ | + | * [[Gramps XML]] file format is robust and well [[Gramps XML#Gramps_XML_Resources|documented]]. |
==Goals== | ==Goals== | ||
Line 10: | Line 12: | ||
The idea of this experimental '''lxml gramplet''' is to provide a way for using basic lxml features with Gramps XML files. | The idea of this experimental '''lxml gramplet''' is to provide a way for using basic lxml features with Gramps XML files. | ||
− | ''XPath'', ''Xslt'', ''RelaxNG | + | ''XPath'', ''Xslt'', ''XML dump'', ''RelaxNG and XSD validations'', can be used and done by lxml, which provides an [http://lxml.de/compatibility.html API very close] to [http://docs.python.org/3/library/xml.etree.elementtree.html etree ElementTree module] from python 2.5 and later. |
− | The experimental '''lxml gramplet''' aims to use these lxml features[1] by parsing a Gramps XML file generated by Gramps 3.3.x and to generate an output sample, using ''open'' [http://www.w3.org/ W3C] standards ([http://www.w3.org/standards/xml/ XML], [http://www.w3.org/standards/webdesign/ Web design], [http://www.w3.org/standards/webofservices/ Web services], etc ...). | + | The experimental '''lxml gramplet''' aims to use these lxml features[1] by parsing a Gramps XML file generated by Gramps 3.4.x (or 3.3.x) and to generate an output sample, using ''open'' [http://www.w3.org/ W3C] standards ([http://www.w3.org/standards/xml/ XML], [http://www.w3.org/standards/webdesign/ Web design], [http://www.w3.org/standards/webofservices/ Web services], etc ...). |
Line 69: | Line 71: | ||
* You can get a copy of this simple ''draft'' from Addon repository: | * You can get a copy of this simple ''draft'' from Addon repository: | ||
− | http:// | + | http://svn.code.sf.net/p/gramps-addons/code/trunk/contrib/lxml |
− | + | * You can also [http://gramps-addons.svn.sourceforge.net/viewvc/gramps-addons/branches/gramps34/download/lxml.addon.tgz download and install it] as 3.4 addon. | |
− | |||
− | * You can also [http://gramps-addons.svn.sourceforge.net/viewvc/gramps-addons/branches/ | ||
Currently, this addon quickly explores multiple ways. Feel free to modify for your own use. | Currently, this addon quickly explores multiple ways. Feel free to modify for your own use. | ||
Line 79: | Line 79: | ||
==Go further== | ==Go further== | ||
− | === | + | ===Bibliography gramplet ?=== |
− | [http://www.giuspen.com/cherrytree CherryTree] is an hierarchical note taking application, featuring rich text and syntax highlighting, storing all the data (including images) in a single '''''xml file''''' with extension ''.ctd'', which has planned to also implement an integration with zotero content. | + | * [http://www.giuspen.com/cherrytree CherryTree] is an hierarchical note taking application, featuring rich text and syntax highlighting, storing all the data (including images) in a single '''''xml file''''' with extension ''.ctd'', which has planned to also implement an integration with [http://www.zotero.org/ zotero] content. |
+ | |||
+ | * [http://zim-wiki.org/index.html Zim] is a graphical text editor used to maintain a collection of wiki pages. All pages you create in zim are saved as plain text files with wiki formatting. This means that you can access your content with any other editor or file manager without being dependent on zim. You can even have your pages in a revision control system like CVS or use a Makefile to compile your notes into a webpage. Any images you add are just image files which are linked from the text files. This means that [http://zim-wiki.org/index.html zim] can call your standard programs to edit images. When you embed an image in a page the context menu for the image will offer to open it with whatever image manipulation programs you have installed. After editing you just reload the page to see the result. See also [http://zim-wiki.org/extras.html third party contributions]. | ||
+ | |||
+ | ===Collaborative indexes=== | ||
+ | |||
+ | * Tiny Tafel [http://en.wikipedia.org/wiki/Tiny_Tafel] | ||
+ | |||
+ | * [[GENDEX]] | ||
+ | |||
+ | * [http://scrapy.org/ Scrapy] is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing. It should support [[Gramps XML]], [[Gramps_4.0_Wiki_Manual_-_Manage_Family_Trees:_CSV_Import_and_Export|Gramps CSV]] and [[GEPS_009:_Import_Export_Merge|Gramps JSON]]. | ||
===Clients library for FamilySearch API=== | ===Clients library for FamilySearch API=== | ||
Line 126: | Line 136: | ||
===Database compare and merge=== | ===Database compare and merge=== | ||
− | * GrampsCompare.py, a python script for comparing data in 2 Gramps | + | * GrampsCompare.py, a python script for comparing data in 2 Gramps XML files. |
+ | |||
+ | source: [http://sourceforge.net/mailarchive/message.php?msg_id=28173190 Archive (Oct 02, 2011) on gramps-devel mailing list] | ||
+ | |||
+ | * [[ImportGramplet]] | ||
− | + | * [http://svn.code.sf.net/p/gramps-addons/code/trunk/contrib/Differences/ Differences report] | |
===Database backend=== | ===Database backend=== | ||
Line 138: | Line 152: | ||
* Gramps Exhibit and experimental phase for [http://members.tele2.nl/m.d.nauta/typeless_data_entry/typeless_data_entry.html typeless data entry]. | * Gramps Exhibit and experimental phase for [http://members.tele2.nl/m.d.nauta/typeless_data_entry/typeless_data_entry.html typeless data entry]. | ||
− | * [http://akara.info/ Akara] is a platform for developing data services available on the Web, using [http://en.wikipedia.org/wiki/Representational_state_transfer REST] architecture. Akara is open source software written in Python and C. eg, [http://recollection.zepheira.com/ Recollection project] for the Library of Congress. See the [http://recollection.zepheira.com/about/userguide/ user guide] or screencasts (''shockwave flash'') [http://outreach.zepheira.com/public/loc/recollection/video/recollection-augmentation.swf], [http://outreach.zepheira.com/public/loc/recollection/video/recollection-intro.swf]. | + | * [http://akara.info/ Akara] is a platform for developing data services available on the Web, using [http://en.wikipedia.org/wiki/Representational_state_transfer REST] architecture. Akara is open source software written in Python and C. eg, [http://recollection.zepheira.com/ Recollection project] for the Library of Congress. See the [http://recollection.zepheira.com/about/userguide/ user guide] or screencasts (''shockwave flash'') [http://outreach.zepheira.com/public/loc/recollection/video/recollection-augmentation.swf], [http://outreach.zepheira.com/public/loc/recollection/video/recollection-intro.swf], [https://www.youtube.com/watch?v=m-TD4jTWn3U]. |
+ | |||
+ | * [[#Collaborative indexes|Scrapy]] | ||
+ | |||
+ | ===Environment=== | ||
+ | |||
+ | * [[Linux_Genealogy_CD#Ways_to_go_.3F|Genealogical ''user'' tablet]] could also provide a portable environment. | ||
+ | |||
+ | * A simple reader with a crossplatform lib: [http://en.wikipedia.org/wiki/QML qml], [http://qt.nokia.com/ qt4], [http://www.gtk.org/ gtk3], [http://kivy.org kivy], [http://pyjs.org/ pyjamas], [[#HTML_class|html5]]; for generating native apps. | ||
===Faceted classification=== | ===Faceted classification=== | ||
− | A [http://en.wikipedia.org/wiki/Faceted_classification faceted classification] system proposed by [http://en.wikipedia.org/wiki/S._R._Ranganathan Shiyali Ramamrita Ranganathan] with the theory "[http://en.wikipedia.org/wiki/Five_laws_of_library_science five laws in library science]". | + | * A [http://en.wikipedia.org/wiki/Faceted_classification faceted classification], [http://unesdoc.unesco.org/Ulis/cgi-bin/ulis.pl?catno=133325&set=4B1BA8F9_1_463&database=ged&gp=0&mode=e&lin=1&ll=f system] proposed by [http://en.wikipedia.org/wiki/S._R._Ranganathan Shiyali Ramamrita Ranganathan] with the theory "[http://en.wikipedia.org/wiki/Five_laws_of_library_science five laws in library science]". See also [http://en.wikipedia.org/wiki/Folksonomy Folksonomy]. |
+ | |||
+ | * [http://pythonhosted.org/Whoosh/ python-whoosh] can provide a simple way for [http://pythonhosted.org/Whoosh/facets.html generating facets in python]. | ||
===HTML class=== | ===HTML class=== | ||
Line 150: | Line 174: | ||
* Gtk3 | * Gtk3 | ||
− | GTK+3 provides an HTML backend that allows GTK applications to run natively within an HTML5 web navigator. | + | GTK+3 provides an [http://git.gnome.org/browse/gtk+/log/?h=broadway HTML backend] that allows GTK applications to run natively within an HTML5 web navigator. |
− | See [http://people.gnome.org/%7Ealexl/broadway-screencast.ogg sample1], [http://youtu.be/AO-qca9ddqg sample2]. | + | See [http://people.gnome.org/%7Ealexl/broadway-screencast.ogg sample1], [http://youtu.be/AO-qca9ddqg sample2], [http://www.youtube.com/watch?v=hhMFD3ZCrIc sample3]. |
===Interface=== | ===Interface=== | ||
Line 159: | Line 183: | ||
* [http://gramps-project.org/2011/01/gramps-mobile-interface-part-i/ Gramps Mobile Interface] for mobile devices. | * [http://gramps-project.org/2011/01/gramps-mobile-interface-part-i/ Gramps Mobile Interface] for mobile devices. | ||
+ | |||
+ | * [[GEPS_017:_Flexible_gen.lib_Interface|Flexible gen.lib interface]] and current [http://www.gramps-project.org/docs/gen/gen_lib.html DB API]. | ||
===Performances=== | ===Performances=== | ||
− | See [[ | + | See [[Gramps_Performance|Gramps performances]] for comparison on large datasets between different Gramps versions. |
===Web applications=== | ===Web applications=== | ||
− | * [[GEPS_013: | + | * [[GEPS_013:_Gramps_Webapp|GEPS 013]] describes a web-based application that runs in your browser, and requires a server. A prototype is now on-line at http://gramps-connect.org/ which is running trunk on a sample database (id=admin1, password=gramps). |
* [[DenominoViso]] plugin for GRAMPS is a third party plugin that creates an interactive graphical representation of a family tree. DenominoViso creates a grapical webpage in SVG/XHTML/javascript. | * [[DenominoViso]] plugin for GRAMPS is a third party plugin that creates an interactive graphical representation of a family tree. DenominoViso creates a grapical webpage in SVG/XHTML/javascript. | ||
+ | |||
+ | * [[Gramps-tweet]], an Addon mashup between Gramps and Twitter. | ||
+ | |||
+ | * [[#Collaborative indexes|Scrapy]] | ||
+ | |||
+ | * [http://www.newsblur.com/ NewsBlur], etc ... | ||
===XQuery=== | ===XQuery=== | ||
− | :"Or something close to SQL like XQuery so you can do querys on | + | :"Or something close to SQL like XQuery so you can do querys on Gramps XML database similar to SQL Query. It can works even in internet browser thru plugins. XML is quite self-explanatory. [http://www.zorba-xquery.com Zorba] provide python bindings for XQuery." |
− | source: [http://sourceforge.net/ | + | ;source: [http://sourceforge.net/p/gramps/mailman/message/23856194/ Archive (Oct 28, 2009) on gramps-user mailing list] |
[[Category:Plugins]] | [[Category:Plugins]] | ||
[[Category:Developers/General]] | [[Category:Developers/General]] | ||
+ | [[Category:Gramplets]] |
Revision as of 01:02, 17 November 2014
This is a Third-party Addon. Please use carefully on data that is backed up, and help make it better by reporting any comments or problems to the author, or issues to the bug tracker |
lxml gramplet is an experimental gramplet working under POSIX platform(s), which reads, writes (not the original one; safe read only state), transforms content of our Gramps XML file on the fly without an import into our database (Gramps session).
Contents
- 1 Dependencies and file format
- 2 Goals
- 3 Screenshots
- 4 Test it
- 5 Go further
- 5.1 Bibliography gramplet ?
- 5.2 Collaborative indexes
- 5.3 Clients library for FamilySearch API
- 5.4 Comments on DB API Idea
- 5.5 Database compare and merge
- 5.6 Database backend
- 5.7 Data transfer
- 5.8 Environment
- 5.9 Faceted classification
- 5.10 HTML class
- 5.11 Interface
- 5.12 Performances
- 5.13 Web applications
- 5.14 XQuery
Dependencies and file format
- lxml is a Pythonic binding for the C libraries libxml2 and libxslt. It is known for good performances by using C-level (Cython).
- Gramps XML file format is robust and well documented.
Goals
The idea of this experimental lxml gramplet is to provide a way for using basic lxml features with Gramps XML files.
XPath, Xslt, XML dump, RelaxNG and XSD validations, can be used and done by lxml, which provides an API very close to etree ElementTree module from python 2.5 and later.
The experimental lxml gramplet aims to use these lxml features[1] by parsing a Gramps XML file generated by Gramps 3.4.x (or 3.3.x) and to generate an output sample, using open W3C standards (XML, Web design, Web services, etc ...).
[1] see also lxml.objectify
Screenshots
- Titles, labels and footer are translated (written on python code).
- Full separation of presentation and content for the generation.
- Local output with custom XML data in buffer and XSLT transformation
- Local output without stylesheet
- View via HTML view
- Pseudo dynamic code generation (xml + xslt = html file)
- Action on surname (sort, remove duplicated)
- Action on place title (sort, enable cross search on place fields)
- Hardcoded list written in python and translated by Gramps into our locale (if translation exists)
Test it
- You can get a copy of this simple draft from Addon repository:
http://svn.code.sf.net/p/gramps-addons/code/trunk/contrib/lxml
- You can also download and install it as 3.4 addon.
Currently, this addon quickly explores multiple ways. Feel free to modify for your own use.
Go further
Bibliography gramplet ?
- CherryTree is an hierarchical note taking application, featuring rich text and syntax highlighting, storing all the data (including images) in a single xml file with extension .ctd, which has planned to also implement an integration with zotero content.
- Zim is a graphical text editor used to maintain a collection of wiki pages. All pages you create in zim are saved as plain text files with wiki formatting. This means that you can access your content with any other editor or file manager without being dependent on zim. You can even have your pages in a revision control system like CVS or use a Makefile to compile your notes into a webpage. Any images you add are just image files which are linked from the text files. This means that zim can call your standard programs to edit images. When you embed an image in a page the context menu for the image will offer to open it with whatever image manipulation programs you have installed. After editing you just reload the page to see the result. See also third party contributions.
Collaborative indexes
- Tiny Tafel [1]
- Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing. It should support Gramps XML, Gramps CSV and Gramps JSON.
Clients library for FamilySearch API
Serialization for C client library or Objective C Client library is done in conjunction with libxml2.
Comments on DB API Idea
I was basically approaching it from the leave gen.lib alone and implement a "fully blown" SimpleAccess-esque solution. At the moment I basically have a 'DB' object which represents an open database. This at the moment is populated from a Gramps XML file. This is then basically stored as lxml.objectify objects. Internally a graph structure is built to represent the linking inside the database (so relationships and ref. integrity is made easier). 'DBItem' objects consist of the 'node' data, the basic save/delete etc... Deleting an event automatically removes all other references to it (which has caught me out previously). class Person(DBItem): DBTYPE = 'person' Basically registers an object that 'wraps' a basic DBItem, but containing useful attributes/methods. So for a person, we can write attributes such as .birth, .mother, .families etc... etc... It can also over-ride how it should be saved/retrieved etc... I chose this approach because it keeps the process incremental. We can still access the 'raw' data in a DBItem for the stuff I'm not caring about at the moment, but someone can write a 'Place' class later for instance. The DB itself is an xpath queryable object (adds a bit of flexibility for selections that don't have convenient attributes as of yet). I'll see if I can get the code example out this week. Anyway, does this seem a reasonable approach?
source: Archive (Dec 07, 2009) on gramps-devel mailing list
Database compare and merge
- GrampsCompare.py, a python script for comparing data in 2 Gramps XML files.
source: Archive (Oct 02, 2011) on gramps-devel mailing list
Database backend
- DB backend for GRAMPS: SQL ?
Data transfer
- Gramps Exhibit and experimental phase for typeless data entry.
- Akara is a platform for developing data services available on the Web, using REST architecture. Akara is open source software written in Python and C. eg, Recollection project for the Library of Congress. See the user guide or screencasts (shockwave flash) [2], [3], [4].
Environment
- Genealogical user tablet could also provide a portable environment.
- A simple reader with a crossplatform lib: qml, qt4, gtk3, kivy, pyjamas, html5; for generating native apps.
Faceted classification
- A faceted classification, system proposed by Shiyali Ramamrita Ranganathan with the theory "five laws in library science". See also Folksonomy.
- python-whoosh can provide a simple way for generating facets in python.
HTML class
- Gramps
Libhtml is an HTML/XML class for Gramps, see API.
- Gtk3
GTK+3 provides an HTML backend that allows GTK applications to run natively within an HTML5 web navigator.
See sample1, sample2, sample3.
Interface
- Alternative interfaces with an experimental gen.lib interface.
- Gramps Mobile Interface for mobile devices.
- Flexible gen.lib interface and current DB API.
Performances
See Gramps performances for comparison on large datasets between different Gramps versions.
Web applications
- GEPS 013 describes a web-based application that runs in your browser, and requires a server. A prototype is now on-line at http://gramps-connect.org/ which is running trunk on a sample database (id=admin1, password=gramps).
- DenominoViso plugin for GRAMPS is a third party plugin that creates an interactive graphical representation of a family tree. DenominoViso creates a grapical webpage in SVG/XHTML/javascript.
- Gramps-tweet, an Addon mashup between Gramps and Twitter.
- NewsBlur, etc ...
XQuery
- "Or something close to SQL like XQuery so you can do querys on Gramps XML database similar to SQL Query. It can works even in internet browser thru plugins. XML is quite self-explanatory. Zorba provide python bindings for XQuery."