16,602
edits
Changes
m
See Current code is now in trunk [http://gramps.svn1791082.sourceforgen4.netnabble.com/viewvc/gramps/branches/geps/gepGEPS023-merged-023into-largetrunk-sources/ SVNtd4157519.html Dec 05, 2011] and[httpFinished://gramps.svn.sourceforge.net/viewvc/gramps/branches/geps/gep-023-large-sources/?view=tar tarball]Gramps 3.4
[http://www.gramps-project.org/bugs/view.php?id=1022 *{{bug|1022}}: Sources dialog hangs for 2 minutes after opening] <br>- This was closed unable to reproduce 2 Feb 2011 [http://www.gramps-project.org/bugs/view.php?id=2918 *{{bug|2918}}: Add Gedcom Source Citation Fields] <br>[http://www.gramps-project.org/bugs/view.php?id=2924 *{{bug|2924}}: Implement GEDCOM EVENT_TYPE_CITED_FROM and ROLE_IN_EVENT] <br>- This was closed as a duplicate of 2918 on 22 Jan 2011[http*{{bug|4491}}://www.gramps-project.org/bugs/view.php?id=4491 4491 (Feature request): Matching source and quality level] <br>- This was closed resolved fixed 28 Jan 2011[http*{{bug|4913}}://www.gramps-project.org/bugs/view.php?id=4913 4913 (Feature request): Additional Event Filters]
no edit summary
Proposed changes for enhancing GRAMPS by enhancing the mechanism for storing data from ‘large’ sources.
== User story (Problem that needs to be solved) ==
MediaRef
Attribute
Repository (Repositories themselves do not have SourceRefs)
Address
Each old SourceRef object should be used to create a new Citation record. The old SourceRef will be replaced by a new CitationBase.
|
+1 PAGE <WHERE_WITHIN_SOURCE> {0:1} p.64
|| SourceRef:Volume/Page becomes Citation:Volume/page
|-
|
+1 EVEN <EVENT_TYPE_CITED_FROM> {0:1} p.49
+2 ROLE <ROLE_IN_EVENT> {0:1} p.61
|| Not supported. See feature request [http://www.gramps-project.org/bugs/view.php?id=2918 2918] and [http://www.gramps-project.org/bugs/view.php?id=2924 2924], (which are mostly duplicates of each other). No change. Any change should be implemented as a separate enhancement, since it is not directly related to the objectives of this GEPS. The data should be stored in the data of the Citation.
|-
|
+1 DATA {0:1}
+2 DATE <ENTRY_RECORDING_DATE> {0:1} p.48
|| SourceRef:Log Date becomes Citation:Log Date
|-
|
+2 TEXT <TEXT_FROM_SOURCE> {0:M} p.63
+3 [CONC|CONT] <TEXT_FROM_SOURCE> {0:M}
|| Not directly supported. Note that in GrampsOn import, one would store the text from source in this is stored as a notewith Note Type 'Source text'. On importexport, the first note of type 'Source text' is output as the 'text from source' should probably be stored as the contents of a Citation:NoteId. On export this GEDCOM field would not be outputNo change.
|-
|
+1 <<MULTIMEDIA_LINK>> {0:M} p.37, 26
|| Not supported at present. Contents stored as the contents of a Citation:MediaRef
|-
|
+1 <<NOTE_STRUCTURE>> {0:M} p.37
|| Contents stored as a note, referenced from the contents SourceRef, which becomes referenced from the Citation. A possible future enhancement (not part of a this GEPS, as it is not directly related to the objectives of this GEPS) would be to support the Data:Value pairs in the SourceRef (which becomes the Citation:NoteId). On export all the Data:Value pairs should probably be concatenated with comma separators into another separate note. On export, the fields in the CitationRef should probably be output as notes.
|-
|
+1 QUAY <CERTAINTY_ASSESSMENT> {0:1} p.43
|| SourceRef:Confidence becomes Citation:Confidence
|-
|
|}
=== User Interface changes ===
Some changes will be needed in the Narrative Web and Simple Database Access functionality.
== Known Issues ==
I believe that all the known issues below can be deferred till after merging GEPS023 into trunk.
==== Issues ====
* Count at bottom right corner of window in citation tree view is incorrect. ''done'' fixed by change to allow filtering of secondary objects [{{Code Commit}}18706]
* Add ID code for citations to the preferences dialog. ''done'' [{{Code Commit}}9cfdeaa8abc99dacf0e76bfa0d2973c1fcb23270]
* Checks and warnings on 'Save' and 'Cancel' in EditCitation are not quite right. ''done '' {{Code Commit}}a4df3034b6924da592d6a543b228fb0c5897aeb0]
==== Deferred ====
* '''src/plugins/import/ImportCsv.py''' Needs to be updated for citations ''done '' [{{Code Commit}}30501a9c5b05f83904abbcedaf14953b222b513a]
* '''src/plugins/export/ExportCsv.py''' Needs to be updated for citations ''done '' [{{Code Commit}}30501a9c5b05f83904abbcedaf14953b222b513a]
* '''src/plugins/import/ImportGeneWeb.py''' Needs changing for citations, but I can't do this as I don't have access to the application : ''done'' [{{Code Commit}}4e18e93ec02abbdb88920674e980964049c8cc6d]
* '''src/plugins/export/ExportGeneWeb.py''' Needs changing for citations, but I can't do this as I don't have access to the application : ''done'' [{{Code Commit}}4e18e93ec02abbdb88920674e980964049c8cc6d]
* '''src/plugins/import/ImportProGen.py''' Needs changing for citations, but I can't do this as I don't have access to the application
* '''src/plugins/tool/TestCaseGenerator.py''' Needs to be updated for citations ''done'' [{{Code Commit}}975f4dc1180ccdc3e0e2d1fd36a8a6794653b0bc]
* '''src/plugins/tool/Check.py''' This has been changed so that it runs, but it needs further enhancement to specifically check citation data ''done'' [{{Code Commit}}975f4dc1180ccdc3e0e2d1fd36a8a6794653b0bc]
* '''src/webapp/grampsdb/models.py''' ''done'' [{{Code Commit}}5892d0b586e7aa8e31b7933799a4baceddfa8d34]
* '''src/webapp/grampsdb/libdjango.py''' ''done'' [{{Code Commit}}5892d0b586e7aa8e31b7933799a4baceddfa8d34]
* '''src/webapp/grampsdb/utils.py''' moved to src/webapp/utils.py? ''done'' [{{Code Commit}}d97ece24a662595cec9ad3688c3dc5a79f3ae6c2]
* '''SimpleAccess, SimpleDoc and SimpleTable''' Needs to be changed for citations, but I am not yet sure how to test these.
* '''po/POTFILES.in and POTFILES.skip''' New files that need translation to be added [http://www.gramps-project.org/bugs/view.php?id=5395#c21977]
* Improve source citation support (import/export) ? see [[Gramps_and_GEDCOM#Information_in_GEDCOM_5.5_not_in_Gramps_.28lost_on_import.29|Gramps and GEDCOM]]
==== Queries ====
* The icon for citation might need to be changed to more closely meet the Tango guidelines. [http://www.gramps-project.org/wiki/index.php?title=GRAMPS_icon_set][http://gramps-project.org/2008/01/30-artwork/][http://sourceforge.net/p/gramps/source/ci/gramps-2.2.1/tree/src/images/sources.svg] ''done'' [{{Code Commit}}a689ac140d0fcf167a050dfd583ede2ca91a1b50]
* Should the default views be changed (e.g. remove the separate source and citation views and provide flat and tree presentations as separate presentations within a single view. Also, should there be a flat sources view).
* In the citationtreeview you can't filter on citation criteria (e.g. confidence) - This is because of the way the views work, the 'Source view' (or citationtreeview) is based on Sources only, so sorting, searching and filtering are just based on sources. If you want to filter the citations, then you have to use the citations view, and the citations filter in that view. ''done'' Improve treebasemodel to allow filtering of secondary object nodes. [{{Code Commit}}809c3bb833e6e94687c72a4aa82e3dbe5ebae19a]
* '''src/gen/proxy/refernced.py''' I have updated this for citations, but it does not appear to be used, so I have not tested it
* '''src/plugins/export/ExportFtree.py''' Not changed for citations because it does not appear to be used
* '''src/plugins/import/ImportGrdb.py''' Don't plan to change this as it appears to be outdated - should it be removed?
* '''src/test/GrampsDb/GrampsDbTestBase.py''' ?? Is this actually used? could be done after merge into citations
* '''src/lib/test/merge_test.py''' I don't plan on upgrading this because it does not seem to be used
* '''src/Merge/merge_ref_test.py''' I don't plan on upgrading this because it does not seem to be used
The test modules have not been updated because of the following comment
[http://gramps.1791082.n4.nabble.com/quot-We-can-always-use-more-developers-quot-td3760985.html#a3762016 The unit tests seem pretty skimpy]
<blockquote>
Yes, somebody started it and it got at the sidetrack. We hardly have time to develop, so unit tests is not something that is in the DNA of our community. Personally I believe that for GUI it is too much work to maintain for a small devel community.
For the library stuff, I would love to see something easy. I however never run unit tests, never heard how to actually run those that are present. The only thing I do for certain parts is add tests at the bottom of the python file that run when the script is executed stand alone (see eg PlaceUtils.py).
During my time at Gramps only those tests have actually catched anything, I never heard of the existing unit tests actually catching any bugs.
Don't understand me wrong, I take testing of my own code very seriously, but I didn't yet see anybody proposing a unit test system for the Gramps codebase that is sufficiently developer friendly so that anybody actually runs them.
</blockquote>
== Completion of the change ==
Although GEPS 023 'only' introduced one new object, it was a major change because it had ramifications throughout all of the Gramps code.
Here are some figures:
* The gep-023-large-sources branch was created on 1 Jul 2011.
* The branch was merged back into trunk on 4 Dec 2011.
* So it took almost 5 months to implement the change.
* 159 files were changed by the time the branch was merged back.
* There were 13,485 lines of u-diff.
I would like to thank Nick Hall for the enormous help and
encouragement I have received from him, especially in implementing the
tree model for two different primary objects which is largely his
code. Without his help, this enhancement would not have been possible.
Also thanks for help, testing and support from Michiel Nauta (advice
on upgrade transactions), John Ralls (fixes to some make files),
Jérôme Rapinat (fixes to translation files etc.), Benny Malengier
(advice on upgrade), Brian Matherly (svn admin), Rob Healey, Gary
Burton, Doug Blank and Cedric Scott and everyone else whose names I
may have missed.
== Changes needed ==
This section provides a guide to the changes that are need to other code (for example add-ons) to interface with citations.
The user interface has been designed on the principle that Aunt Martha
should not need to know or even be aware that citations have changed.
Hence she will see a Source navigation with similar information to the
current view. There will be a disclosure triangle, but she can ignore
that. When she wants to add a source to a person or event, she clicks
on the add or add existing button, and gets a similar choice of source
as at present. (Again she can ignore the disclosure triangles). The
editor that appears is then very similar to the existing editor. Only
if a user particularly wants to share citations between objects do
they need to be aware of the change. The change is also intended to be
entirely compatible with GEDCOM.
Simple access to, and updating of, citations or sources does not
involve very extensive changes to existing code. The database changes,
the various editors, filters and proxies etc. have involved more
extensive changes.
The Citation primary object corresponds pretty closely to the
SourceRef secondary object. Data in the SourceRef is mapped to the
Citation. A Citation has the additional possibility of having media
objects attached (which is not possible with SourceRefs). A Citation
points to exactly one Source (the code may assume that here is always
a source object present).
The 'shape' of existing objects is unchanged. The change is that
SourceBase is replaced by CitationBase.
For creating data, where you previously created a SourceRef, you now
create a Citation object, and set_reference_handle() to point to the
Source object. There is a difference that you then have to do a
db.add_citation(citation, transaction). You then do an
obj.add_citation(citation_handle) instead of an
obj.add_source_reference(sourcerefobj) to add the SourceRef/Citation
to the object.
An example of the changes is:
{{Code Commit}}6ee1af893fbe805a4c67d672d5175f78fcdc6596
As far as data access is concerned, you do obj.get_citation_list()
instead of obj.get_source_references() to get a list of Citation
handles/SourceRefs respectively. With the citation, you then need to
do a db.get_citation_from_handle(citation_handle). Having got the
object, you do get_reference_handle() to get the source object handle.
An example of the changes is:
{{Code Commit}}e4383f0b46b5be1258e5f7e2caef2c4c48fa39b9
Note 1. The names of the functions and the return values are mostly
chosen to match the way notes are attached to objects, so
obj.add_citation(citation_handle) corresponds with add_note - both
taking handles as parameters, and obj.get_citation_list() corresponds
with get_note_list() - both returning handles.
Note 2. I am aware of PEP 8 and pylint and have followed them for new
code. However, where I have just been modifying existing code, or have
largely copied some existing code, I have retained the style of the
existing code, for example in terms of code layout, naming conventions
and the number of parameters or methods in a class. This may result in
a lowered pylint score.
[N.B. This is only a guide, no guarantee!]
=== Bug tracker ===
=== Others interfaces ===
*[http://www.ancestralquestonline.com/desbk/aq_sources.htm Source edition into Ancestral Quest]
*[http://west-penwith.org.uk/misc/ftmsour.htm Source Citations in FTW], now [http://www.geneamusings.com/2011/01/peeking-at-family-tree-maker-2011.html Source citation into Family Tree Maker]*[http://west-penwith.org.uk/misc/ftmsour.htm Source Citations in FTW]
*[http://www.geneamusings.com/2011/01/peeking-at-legacy-family-tree-75-source.html Source citation into Legacy Family Tree]
=== Templates ===
*[http://sourcetemplates.org/ Source templates project]
**[http://sourcetemplates.org/details.php details]
=== Other ===
*[http://bettergedcom.wikispaces.com/Sources+and+Citations Sources and Citations]
*[[GEPS_015:_Repository_Research_Support|GEPS 015:Repository Research Support]]
*[[GEPS_018:_Evidence_style_sources|GEPS 018:Evidence style sources]]
*[[GEPS_024:_Certificates|GEPS 024:Certificates]]
[[Category:GEPS|S]]
[[Category:Developers/Design]]
