Difference between revisions of "GEPS 026: Replace 'make' for Gramps build"

From Gramps
Jump to: navigation, search
m (Added gramps/data directory to list of package_data.)
(Remove link to geps branch)
(9 intermediate revisions by 4 users not shown)
Line 1: Line 1:
Move Gramps away from using [http://en.wikipedia.org/wiki/Make_%28software%29 Makefiles] and change to using Pythons setup.py and/or Distutils [http://sourceforge.net/mailarchive/message.php?msg_id=28984484][http://sourceforge.net/mailarchive/message.php?msg_id=28969888][http://sourceforge.net/mailarchive/message.php?msg_id=27984817]{{bug|2621}}
+
Move Gramps away from using [http://en.wikipedia.org/wiki/Make_%28software%29 Makefiles] and change to using Pythons setup.py and/or Distutils {{bug|2621}} {{bug|5743}} [http://sourceforge.net/mailarchive/message.php?msg_id=28984484][http://sourceforge.net/mailarchive/message.php?msg_id=28969888][http://sourceforge.net/mailarchive/message.php?msg_id=27984817]
 
 
*[https://gramps.svn.sourceforge.net/svnroot/gramps/branches/geps/gep-026-replace-make/ SVN]
 
  
 
==Purpose==
 
==Purpose==
Line 12: Line 10:
 
# An increasing number of projects that need GNU Make switch to [http://www.cmake.org/ CMake], ..., we should not stay with old technology just because it was already present
 
# An increasing number of projects that need GNU Make switch to [http://www.cmake.org/ CMake], ..., we should not stay with old technology just because it was already present
  
==Distutils2==
+
==Posibilities==
 +
===Distutils===
 +
Common way of distributing python. This is currently in gramps40 and trunk.  It uses the setup.py and MANIFEST.in files.
 +
 
 +
To create a source distribution run:
 +
 
 +
python setup.py sdist
 +
 
 +
It creates a tar file in dist directory.  It also creates a MANIFEST file that contains files in distribution.
 +
 
 +
To build run:
 +
 
 +
python setup.py build
 +
 
 +
Files are created in build directory.
 +
 
 +
To install run:
 +
 
 +
python setup.py install
 +
 
 +
For testing it is convenient to install to a different root directory and disable execution of post-install mime processing.
 +
 
 +
python setup.py install --root ~/test --enable-packager-mode
 +
 
 +
===Packaging (Distutils2)===
  
 +
The way of the future. At this time it seems not ready yet for use in Gramps.
 
We need to create a setup.cfg file which defines the following:
 
We need to create a setup.cfg file which defines the following:
  
Line 42: Line 65:
 
A list of file specifications of files that need to be included in the distribution, but should not be installed.  This provides an alternative to the MANIFEST.in file, and should be easier to maintain.  Another alternative would be a 'manifest-builder', which we should investigate.  This could automatically include all files that are under version control.
 
A list of file specifications of files that need to be included in the distribution, but should not be installed.  This provides an alternative to the MANIFEST.in file, and should be easier to maintain.  Another alternative would be a 'manifest-builder', which we should investigate.  This could automatically include all files that are under version control.
  
Note:  Issue [http://bugs.python.org/issue14691 14691] prevents specifying a manifest builder in the config file.  There is a patch available to fix this.
+
Note:  Issue [http://bugs.python.org/issue14691 14691] prevents specifying a manifest builder in the config file.  There is a patch available to fix this.  This issue has been resolved/ closed on 2012-04-29.
  
distutils2 provides support for customisation via command hooks.  Our translation build code should be run via a build command hook. This should allow us to access the build directory and add files to the 'resources' as necessary.
+
distutils2 provides support for customisation via command hooks.  Our translation build code, compiling manual page, and XDG files (gramps.desktop, gramps.keys, and gramps.xml), and (holidays.xml and tips.xml) files will be created by these command hooks in the correct sequences.
 +
* They are being handled by the setup_custom.py file under:
 +
# build_intl()
 +
# build_trans()
 +
# build_man() functions.
  
==Issues==
+
====Issues====
  
 
* [http://www.python.org/dev/peps/pep-0345/ PEP 345]: Metadata for Python Software Packages 1.2
 
* [http://www.python.org/dev/peps/pep-0345/ PEP 345]: Metadata for Python Software Packages 1.2
Line 53: Line 80:
 
* [http://www.python.org/dev/peps/pep-0390/ PEP 390]: Static metadata for Distutils
 
* [http://www.python.org/dev/peps/pep-0390/ PEP 390]: Static metadata for Distutils
  
==Bugs==
+
====Bugs====
  
 
* [http://bugs.python.org/issue5302 5302]: Allow package_data specs/globs to match directories
 
* [http://bugs.python.org/issue5302 5302]: Allow package_data specs/globs to match directories
Line 61: Line 88:
 
* [http://bugs.python.org/issue13463 13463]: Fix parsing of package_data
 
* [http://bugs.python.org/issue13463 13463]: Fix parsing of package_data
 
* [http://bugs.python.org/issue14549 14549]: Recursive inclusion of packages
 
* [http://bugs.python.org/issue14549 14549]: Recursive inclusion of packages
* [http://bugs.python.org/issue14651 14651]: pysetup run cmd can't handle option values in the setup.cfg
+
* [http://bugs.python.org/issue14651 14651]: pysetup run cmd can't handle option values in the setup.cfg file.  Patch is available at the bug tracker.
 +
 
 +
====Details====
 +
This now works in trunk.  Installation instructions can be found [http://www.gramps-project.org/bugs/view.php?id=6093 here].
 +
 
 +
To create a source distribution run:
 +
 
 +
pysetup run sdist
 +
 
 +
To build run:
 +
 
 +
pysetup run build
 +
 
 +
This will fail without the patch (bug #14651).
 +
 
 +
To install run:
 +
 
 +
pysetup run install_dist
 +
 
 +
Testing is a pain because of a bug with --dry-run and a bug with --install-data.  I suggest using the --home option.
 +
Also, there is not an --enable-packager-mode option for distutils2 yet.
 +
 
 +
=== Distribute (Setuptools)===
 +
We don't consider this.
 +
 
 +
== Decision ==
 +
For now, we will use distutils and deprecate autotools.
 +
 
  
 
[[Category:GEPS|D]]
 
[[Category:GEPS|D]]

Revision as of 15:43, 7 November 2013

Move Gramps away from using Makefiles and change to using Pythons setup.py and/or Distutils 2621 5743 [1][2][3]

Purpose

  1. The core Gramps developers do not know GNU Make, which was the case also in the past. Somebody of a company with expertise contributed the make system to Gramps. (Actually, I'm pretty fluent with autotools. John Ralls)
  2. With GNU Make developers need to update makefiles when adding and changing files.
    *With setup.cfg/ DistUtils2, developers only need to worry about is creating quality Python code. Searching and adding files to Makefiles is no longer needed as it is handled dynamically now.
  3. With GNU Make from time to time it happens that a new file is forgotten to be added in a release, causing maintainance overhead for the release manager.
  4. Gramps is pure python, we don't need a makefile system checking for a lot of things it is checking for now.
  5. We need less dependencies for installing Gramps in linux.
    *Already on windows makefiles are not used.
  6. An increasing number of projects that need GNU Make switch to CMake, ..., we should not stay with old technology just because it was already present

Posibilities

Distutils

Common way of distributing python. This is currently in gramps40 and trunk. It uses the setup.py and MANIFEST.in files.

To create a source distribution run:

python setup.py sdist

It creates a tar file in dist directory. It also creates a MANIFEST file that contains files in distribution.

To build run:

python setup.py build

Files are created in build directory.

To install run:

python setup.py install

For testing it is convenient to install to a different root directory and disable execution of post-install mime processing.

python setup.py install --root ~/test --enable-packager-mode

Packaging (Distutils2)

The way of the future. At this time it seems not ready yet for use in Gramps. We need to create a setup.cfg file which defines the following:

  • packages

A list of packages that should be installed in the gramps directory under python site-packages. This should be a list of all packages, but excluding test and experimental packages. We can generate this dynamically to ease maintenance. There is a function provided to do this.

Note: Issue 14549 requests an enhancement to this functionality.

  • package_data

A list of file specifications of all non-package files that should also be installed in the gramps directory under python site-packages. This should specify all files under the data, glade, images and plugins directories.

Note: Issues 5302 and 13463 cause a problem with this at the moment.

  • resources

A list of file specifications of files that need to be installed outside the gramps directory under python site-packages. Targets for files include 'categories' which are OS independent.

Note: Issue 12393 requests support for extensible categories which could be useful to us.

  • scripts

A list of scripts. Just gramps.sh in our case.

  • extra_files

A list of file specifications of files that need to be included in the distribution, but should not be installed. This provides an alternative to the MANIFEST.in file, and should be easier to maintain. Another alternative would be a 'manifest-builder', which we should investigate. This could automatically include all files that are under version control.

Note: Issue 14691 prevents specifying a manifest builder in the config file. There is a patch available to fix this. This issue has been resolved/ closed on 2012-04-29.

distutils2 provides support for customisation via command hooks. Our translation build code, compiling manual page, and XDG files (gramps.desktop, gramps.keys, and gramps.xml), and (holidays.xml and tips.xml) files will be created by these command hooks in the correct sequences.

  • They are being handled by the setup_custom.py file under:
  1. build_intl()
  2. build_trans()
  3. build_man() functions.

Issues

  • PEP 345: Metadata for Python Software Packages 1.2
  • PEP 376: Database of Installed Python Distributions
  • PEP 386: Changing the version comparison module in Distutils
  • PEP 390: Static metadata for Distutils

Bugs

  • 5302: Allow package_data specs/globs to match directories
  • 8501: --dry-run option doesn't work
  • 12830: --install-data doesn't effect resources destination
  • 12393: Packaging should provide support for extensible categories
  • 13463: Fix parsing of package_data
  • 14549: Recursive inclusion of packages
  • 14651: pysetup run cmd can't handle option values in the setup.cfg file. Patch is available at the bug tracker.

Details

This now works in trunk. Installation instructions can be found here.

To create a source distribution run:

pysetup run sdist

To build run:

pysetup run build

This will fail without the patch (bug #14651).

To install run:

pysetup run install_dist

Testing is a pain because of a bug with --dry-run and a bug with --install-data. I suggest using the --home option. Also, there is not an --enable-packager-mode option for distutils2 yet.

Distribute (Setuptools)

We don't consider this.

Decision

For now, we will use distutils and deprecate autotools.