Restructuring the database objects, Part III

At last I’m ready to release the fruits of my labor. Today I will be making two large commits to gramps trunk:

  1. Changes to the basic database objects and other modules to activate the new code.
  2. Changes to the proxy database objects to (hopefully) reduce their processing time.

The first group “flips the switch,” so to speak, on the new modules read.py and write.py. These contain the methods formerly spread between base.py and dbdir.py. The modified base.py is now strictly abstract (or aspires to be!) As their names imply, read.py contains methods necessary to read the database; write.py contains methods used to add, update and delete database records. I’ve been working with these for more than a month and have tested them on databases large and small, running through all typical functions from creation to reporting, import to export, and utility functions like check and repair, rebuild secondary indices and rebuild reference maps. Now is the time to turn it all loose and find and fix the breakages I missed!

The second group of changes concerns the proxy databases — work I began then suspended until the larger work neared completion. I hope that you see performance improvements from these changes. I know that I do. One interesting thing I found is that, when considering a record for inclusion, each proxy object would call the get_”object”_from_handle methods and check the results. This often meant double calls, since proxy a might be running through the objects in its database, which can cause it to call proxy b first. But it happens that if proxy b is already responding to an iterator call, it has already filtered the results it is passing to proxy a, so it is redundant for proxy a to call proxy b again for each record. I circumvented this by implementing a dynamic get_unfiltered_”object” method, that drills down to the underlying “real” database. You can see how it is used in the new predicate functions (e.g. “include_person”). The new methods are built on-the-fly using a __getattr__ hook.

The committed revisions for these two updates are 13139 and 13140.

Join the Conversation!