Difference between revisions of "Unit Test Quickstart"

From Gramps
Jump to: navigation, search
m (Add a category)
(unittest.mock)
(14 intermediate revisions by 5 users not shown)
Line 1: Line 1:
 
{{languages|Unit Test Quickstart}}
 
{{languages|Unit Test Quickstart}}
  
This page gives some simple recipes and tips for writing unit tests in the Gramps source code tree.
+
Simple recipes and tips for writing unit tests in the Gramps source code tree.
  
 
=== First, some general procedural and structural notes ===
 
=== First, some general procedural and structural notes ===
Line 10: Line 10:
 
* the leading part of the test will commonly be named the same as the basename (or a variant) of the module under test, leading, for example, to the following files
 
* the leading part of the test will commonly be named the same as the basename (or a variant) of the module under test, leading, for example, to the following files
  
  src/A/B/module.py
+
  gramps/A/B/module.py
  src/A/B/test/module_test.py
+
  gramps/A/B/test/module_test.py
  
 
* the test subdirectory can contain data or helper programs as required
 
* the test subdirectory can contain data or helper programs as required
Line 17: Line 17:
 
* the test module can create and delete test data during execution. Commonly this would go in a deeper subdir, named to avoid collision with other test programs.
 
* the test module can create and delete test data during execution. Commonly this would go in a deeper subdir, named to avoid collision with other test programs.
  
...more to come...
+
{{stub}}
 +
 
 +
=== Running Unit Tests ===
 +
 
 +
A single unit test can be run from the top-level Gramps directory.
 +
 
 +
GRAMPS_RESOURCES=. python -m unittest <test_module>
 +
 
 +
for example:
 +
 
 +
GRAMPS_RESOURCES=. python -m unittest gramps.gen.lib.test.date_test
 +
 
 +
To run all the unit tests use:
 +
 
 +
GRAMPS_RESOURCES=. python -m unittest discover -p '*_test.py'
 +
 
 +
For verbose output use the -v flag:
 +
 
 +
GRAMPS_RESOURCES=. python -m unittest discover -p '*_test.py' -v
 +
 
 +
For convenience, all the unit tests can be run from the setup.py script using:
 +
 
 +
python setup.py test
 +
 
 +
For verbose output use:
 +
 
 +
python setup.py --verbose test
  
 
=== Simple Code Recipe ===
 
=== Simple Code Recipe ===
Line 27: Line 53:
 
import unittest
 
import unittest
  
# import this test support utility before importing the module under test
+
# import the module under test
from test import test_util    # test is a GRAMPS module
+
import ..MyModule                # use your module name
test_util.path_append_parent() # enables the following import
 
import MyModule                # use your module name
 
 
   
 
   
# look in gramps/src/test/test_util.py for other conveniences (and suggest more ideas)
+
# find the data directory
this_dir = test_util.abspath()
+
this_dir = os.path.dirname(__file___)
data_dir = test_util.make_subdir("MyModule_test_data")
+
data_dir = os.path.join(this_dir, 'data')
 
   
 
   
 
# unittest requires a TestCase class containing test function members  
 
# unittest requires a TestCase class containing test function members  
Line 44: Line 68:
 
         ...
 
         ...
  
     def test_function_x(self):
+
     def test_action_x_leads_to_y(self):
 
         ..do stuff..
 
         ..do stuff..
 
         self.assertTrue(expression, "message to display on failure")
 
         self.assertTrue(expression, "message to display on failure")
Line 70: Line 94:
 
* until happy
 
* until happy
 
* check-in  your module code and test code
 
* check-in  your module code and test code
 +
 +
This works in gramps40/trunk from the toplevel dir to run the existing exportvcard_test.py:
 +
GRAMPS_RESOURCES=$PWD PYTHONPATH=$PWD:$PWD/gramps python gramps/plugins/export/test/exportvcard_test.py
 +
 +
See also [[Testing Gramps]].
 +
 +
=== Mocking external dependencies with unittest.mock ===
 +
When you develop a class or a method with external dependencies, you often would like to provide test coverage just to the developed code proper, not the external dependencies it excercised. For instance, if your code calls sys.stderr.write, there is no need to check that sys.stderr.write actually outputs anything on the system standard error file descriptor. Rather, you'd like to know that you code calls the write method and what arguments get passed.
 +
 +
Python3 provides [http://docs.python.org/dev/library/unittest.mock unittest.mock] module for such tasks. Before Python 3.3, you can install the "mock" module that back-ports this functionality, available as "python-mock" package on Ubuntu.
 +
 +
Gramps tests using mocks will be skipped if the module is not available.
  
 
=== Further Information ===
 
=== Further Information ===
 
* other pages needed
 
* other pages needed
 
* related topics
 
* related topics
 +
** [[Programming_Guidelines#Pylint|Programming guidelines - pylint]]
 
* references, tutorials, etc
 
* references, tutorials, etc
 
* ...TBD...
 
* ...TBD...
  
 
Perhaps: Just add questions and suggestions to this wiki page!
 
Perhaps: Just add questions and suggestions to this wiki page!
 
  
 
[[Category:Developers/Reference]]
 
[[Category:Developers/Reference]]
 +
[[Category:Developers/Quality Assurance]]

Revision as of 11:44, 26 August 2013

Simple recipes and tips for writing unit tests in the Gramps source code tree.

First, some general procedural and structural notes

  • unit tests are usually created to verify correct functional behavior of a single module of source code.
  • test code goes in a test subdirectory of the subject module's directory
  • the test module should be named with a suffix _test so that it can be found by automatic regression test tools.
  • the leading part of the test will commonly be named the same as the basename (or a variant) of the module under test, leading, for example, to the following files
gramps/A/B/module.py
gramps/A/B/test/module_test.py
  • the test subdirectory can contain data or helper programs as required
  • the test and such supplementary elements that are persistent will be maintained as part of the source control system.
  • the test module can create and delete test data during execution. Commonly this would go in a deeper subdir, named to avoid collision with other test programs.
Gramps-notes.png

This article's content is incomplete or a placeholder stub.
Please update or expand this section.


Running Unit Tests

A single unit test can be run from the top-level Gramps directory.

GRAMPS_RESOURCES=. python -m unittest <test_module>

for example:

GRAMPS_RESOURCES=. python -m unittest gramps.gen.lib.test.date_test

To run all the unit tests use:

GRAMPS_RESOURCES=. python -m unittest discover -p '*_test.py'

For verbose output use the -v flag:

GRAMPS_RESOURCES=. python -m unittest discover -p '*_test.py' -v

For convenience, all the unit tests can be run from the setup.py script using:

python setup.py test

For verbose output use:

python setup.py --verbose test

Simple Code Recipe

There are only a few firm requirements to fit into the framework. Here is a simple test module that may be considered something of a template.

import unittest

# import the module under test
import ..MyModule                # use your module name
 
# find the data directory
this_dir = os.path.dirname(__file___)
data_dir = os.path.join(this_dir, 'data')
 
# unittest requires a TestCase class containing test function members 
# optional setUp() and tearDown() functions can perform pre/post test housekeeping
class Test_top(unittest.TestCase):
     def setUp(self):
         ...
     def tearDown(self):
         ...

     def test_action_x_leads_to_y(self):
         ..do stuff..
         self.assertTrue(expression, "message to display on failure")
         #see other assert and fail functions in the unittest module

     ..more defs for more tests, more classes for possible grouping logic..

if __name__ == "__main__":
    unittest.main()

Using Your Unit Test

  • create a module_test.py if and whenever it seems like it might be useful
  • create a test subdir if one is not already there
  • one practice might be as follows
cd ...gramps/src
export PYTHONPATH=`pwd`
cd A/B/test
gvim +ba ../module.py module_test.py
  • repeat
    • do some editing
    • do some testing as follows (optional -v shows extra progress messages)
 python module_test.py -v
  • until happy
  • check-in your module code and test code

This works in gramps40/trunk from the toplevel dir to run the existing exportvcard_test.py:

GRAMPS_RESOURCES=$PWD PYTHONPATH=$PWD:$PWD/gramps python gramps/plugins/export/test/exportvcard_test.py

See also Testing Gramps.

Mocking external dependencies with unittest.mock

When you develop a class or a method with external dependencies, you often would like to provide test coverage just to the developed code proper, not the external dependencies it excercised. For instance, if your code calls sys.stderr.write, there is no need to check that sys.stderr.write actually outputs anything on the system standard error file descriptor. Rather, you'd like to know that you code calls the write method and what arguments get passed.

Python3 provides unittest.mock module for such tasks. Before Python 3.3, you can install the "mock" module that back-ports this functionality, available as "python-mock" package on Ubuntu.

Gramps tests using mocks will be skipped if the module is not available.

Further Information

Perhaps: Just add questions and suggestions to this wiki page!