Open main menu

Gramps β

Changes

Testing Gramps

4,292 bytes added, 23:37, 7 July 2015
→‎Possible improvements
=Currently used Test status=As of Jan 2015 we don't have a record of tests executed, the platforms and environments they were run upon, and what code they covered. The only indirect evidence is available in open bugs, when people care to fill in these details. :-( ==Possible improvements==* Unify running all the tests, in- or out- tree** Try switching the feature tests and non-automated unit tests in test/ from our runner scripts to [https://docs.python.org/library/unittest.html#test-discovery python native unittest discovery mechanism]. ** We have a stale [https://github.com/gramps-project/gramps/blob/master/gramps/test/regrtest.py gramps/test/regrtest.py] runner, notable for logging init. - should we revive that, or maybe integrate into setup.py test runner?* [https://wiki.python.org/moin/CodeCoverage Coverage analysis]* [http://docs.python-guide.org/en/latest/scenarios/ci/ Continuous] test status report, coverage, automatic deployment into win/mac/linux VMs (needs server capacity to be hosted online)(I can dream, can't I?){{bug|8294}}* Automated regression tests for our GUI. The following links look interesting:** [https://wiki.python.org/moin/PythonTestingToolsTaxonomy#GUI_Testing_Tools Python Wiki:GUI Testing Tools]** [http://unpythonic.blogspot.co.il/2007/03/unit-testing-pygtk.html Unit Testing PyGTK]** [http://ldtp.freedesktop.org/wiki/ Cross Platform GUI Test Automation tool] =Tests and frameworksused=== Manual test plan ==See [http://sourceforge.net/p/gramps/source/ci/master/tree/TestPlan.txt TestPlan.txt] in gramps toplevel. (I believe this is only done at a major release (like Gramps version 4.0.0)). == Specialized scripts for testing ==See more specialized scripts in [http://sourceforge.net/p/gramps/source/ci/master/tree/test/ test/], status unknown. Contents of [http://sourceforge.net/p/gramps/source/ci/master/tree/test/ test/] :* GrampsLogger/** ErrorReportAssistant_Test.py** GtkHandler_Test.py* LosHawlos_bsddbtest.py* LosHawlos_dbtest.py* RunAllTests.py* dates.sh* det_ancestor_report.sh* det_descendant_report.sh* impex.sh* runtest.sh* tools.sh === testing of reports ======= test/runtest.sh ==== '''[http://sourceforge.net/p/gramps/source/ci/master/tree/test/runtest.sh test/runtest.sh]''' - Report test for Gramps: Generate every report in every format. Runs all possible reports using the report cli interface, based on the example.gramps database. This test is not fully self-contained, in that it depends on various environment settings, such as your locale, your preferred name display formats, and your report options. Last, but not the least, the verification of the resulting reports is entirely manual.
Bugs tagged as [http://www.gramps-project.org/bugs/search.php?tag_string=found-by-runtest.sh found-by-runtest.sh]
== other report testing =test/impex.sh===See more specialized scripts in '''[http://sourceforge.net/p/gramps/source/ci/master/tree/test/, status unknownimpex. ==sh test/impex.sh==]''' - Import/export test for GRAMPSGramps.
From the file header:
== Unit testing ==
=== test/RunAllTests.py ===
'''[http://sourceforge.net/p/gramps/source/ci/master/tree/test/RunAllTests.py test/RunAllTests.py]''' - Testing framework for performing a variety of unittests for Gramps. Runs out-of-tree (not in gramps/) testing code, by looking for any test/*_Test.py files and executing the test suites therein. See the current code in test/*_Test.py for example and python standard unittest docs.
{{man note|Starting with gramps40 gramps4.x branch, |these tests include non-automated unit tests only. The automated unit tests are all under gramps/.}}
Bugs tagged as [http://www.gramps-project.org/bugs/search.php?tag_string=found-by-RunAllTests.py found-by-RunAllTests.py]
{{man warn|GtkHandler testing code pops up the GRAMPS Gramps error dialog, |but this is actually for testing the error reporting itself. Don't be scared by the dialog, it's expected. Your manual work is required to close the dialogs with the "Cancel" button. The relevant tests still pass (unless there's another bug there)... One test currently fails, see {{bug|6940}}.
=== unit tests in the main tree ===
python setup.py test
See [[Unit Test Quickstart]] for detailed running instructions.
==== using - python setup.py test ==== python setup.py test {| {{prettytable}}class="wikitable sortable"
! File
! Test
! Status
! Comments
|-
|gramps/cli/test/cli_test.py
|3
|bgcolor="#80ff80"| OK
|
|-
|gramps/cli/test/user_test.py
|8
|bgcolor="#80ff80"| OK
|3 tests(6 test require [[Unit_Test_Quickstart#Mocking_external_dependencies_with_unittest.mock| mocking]] to run)
|-
|gramps/gen/test/config_test.py
|1
|bgcolor="#80ff80"| OK
|
|-
|gramps/gen/test/constfunc_test.py
|1
|bgcolor="#80ff80"| OK
|(linux only, skipped elsewhere)
|-
|gramps/gen/test/user_test.py
|1
|bgcolor="#80ff80"| OK
|See #{{bug|7013}} for context
|-
|gramps/gen/datehandler/test/datehandler_test.py
|9
|bgcolor="#80ff80"| OK
|1 test
|-
|gramps/gen/db/test/cursor_test.py
|4
|bgcolor="#80ff80"| OK
|4 tests, 2 skipped
|-
|gramps/gen/db/test/db_test.py
|6
|bgcolor="#80ff80"| OK
|6 tests
|-
|gramps/gen/db/test/reference_map_test.py
|5
|bgcolor="#80ff80"| OK
|5 tests
|-
|gramps/gen/lib/test/date_test.py
|8
|bgcolor="#80ff80"| OK
|8 tests(locale-based, OK for en, fails for some other locales)
|-
|gramps/gen/lib/test/grampstype_test.py
|3
|bgcolor="#80ff80"| OK
|3 tests
|-
|gramps/gen/lib/test/merge_test.py
|201|bgcolor="#80ff80ff8080"| OKFAIL|201 tests2 failed skipped ({{bug|7027}}- Bug shows resolved)
|-
|gramps/gen/merge/test/merge_ref_test.py
|53|bgcolor="#ff808080ff80"| FAILOK|53 tests, 2 failures (bug #{{bug|6960}})
|-
|gramps/gen/utils/test/callback_test.py
|10
|bgcolor="#80ff80"| OK
|10 tests
|-
|gramps/gen/utils/test/keyword_test.py
|4
|bgcolor="#80ff80"| OK
|4 tests
|-
|gramps/gen/utils/test/place_test.py
|28
|bgcolor="#ff8080"| FAIL
|28 tests, 4 failures(bug #{{bug|7044}} - Bug shows resolved)
|-
|gramps/gui/logger/test/rotate_handler_test.py
|2
|bgcolor="#80ff80"| OK
|
|-
|gramps/gui/test/user_test.py
|1
|bgcolor="#80ff80"| OK
|2 tests(requires mocking to run)
|-
|gramps/plugins/export/test/exportvcard_test.py
|28
|bgcolor="#80ff80"| OK
|28 tests
|-
|gramps/plugins/importer/test/importvcard_test.py
|66
|bgcolor="#80ff80"| OK
|66 tests|-|gramps/plugins/importer/test/importxml_test.py|4|bgcolor="#80ff80"| OK||-|gramps/plugins/importer/test/importgeneweb_test.py|2|bgcolor="#80ff80"| OK|
|-
|gramps/test/test/gedread_util_test.py
|
|
|No longer required?
|-
|gramps/test/test/test_util_test.py
|
|
|No longer required?
|-
|gramps/webapp/grampsdb/view/png.py
|
|
|
|}
There is also semi{{man note|Note from Nick Hall|Some of the merge tests are slow because they actually run Gramps from the command line to import and export files.<br><br>Extra packages and add-interactive testing via __main__ in ons are required to run some code:* Relationship calculator testing* more!of the tests. From memory, you will need the CliMerge and ExportRaw add-ons, and the libxml2, libxslt and mocking python packages. Some of these may not be available for python3 yet.}}
== Manual test plan ==See TestPlan.txt in gramps toplevel. I believe this is only done at a major release (like 4.0.0).semi-interactive ====
=Test status=We currently don't have a record of tests executed, the platforms and environments they were run upon, and what code they covered. The only indirect evidence There is available also semi-interactive testing via __main__ in open bugs, when people care to fill in these details. some code:-(
{| class=Possible improvements"wikitable sortable"! File! Status! Comments|-|[http://sourceforge.net/p/gramps/source/ci/master/tree/gramps/gen/relationship.py#l1889 gramps/gen/relationship.py]|bgcolor="#ff8080"| To Do|Relationship calculator|-|gramps/gen/db/bsddbtxn.py|bgcolor="#ff8080"| To Do||-|gramps/gen/db/txn.py|bgcolor="#ff8080"| To Do||-|gramps/gen/db/undoredo.py|bgcolor="#ff8080"| To Do||-|gramps/gen/db/write.py|bgcolor="#ff8080"| To Do||-|gramps/gen/db/txn.py|bgcolor="#ff8080"| To Do||-|gramps/gen/lib/styledtext.py|bgcolor="#ff8080"| To Do||-|gramps/gen/utils/place.py|bgcolor="#80ff80"| OK|Done|-|gramps/gen/utils/docgen/odstab.py|bgcolor="#ff8080"| To Do||-|gramps/gen/utils/docgen/csvtab.py|bgcolor="#ff8080"| To Do||-|gramps/gui/ddtargets.py|bgcolor="#ff8080"| To Do* unify |Not worth running all the tests, in?|- or out|gramps/gui/widgets/undoablebuffer.py|bgcolor="#ff8080"| To Do|Not worth running?|- tree** Try switching the feature tests and non|gramps/gui/dialog.py|bgcolor="#ff8080"| To Do|Interactive|-automated unit tests in test|gramps/ from our runner scripts to [http:gui/widgets/docsvalidatedmaskedentry.pythonpy|bgcolor="#ff8080"| To Do|Interactive|-|gramps/gui/widgets/progressdialog.orgpy|bgcolor="#ff8080"| To Do|Interactive|-|gramps/2gui/librarywidgets/unitteststatusbar.htmlpy|bgcolor="#testff8080"| To Do|Interactive|-discovery python native unittest discovery mechanism]|gramps/plugins/lib/libsubstkeyword. py|bgcolor="#ff8080"| To Do||-|gramps/plugins/rel/rel_* coverage analysis.py|bgcolor="#ff8080"| To Do|Relationship calculator plugins|-* (needs server capacity to be hosted online) continous test status report, coverage, automatic deployment into win|gramps/plugins/mactool/linux VMs (I can dream, can't I?)phpgedviewconnector.py|bgcolor="#ff8080"| To Do||-|}
=See also=
* [[Unit Test Quickstart]]
* [[Test_date_handlers#TO_CHECK]]
* [[Gramps_Performance#The_Test_Results]]
[[Category:Developers/Reference]]
[[Category:Developers/Quality Assurance]]
Retrieved from "https://gramps-project.org/wiki/index.php/Special:MobileDiff/46332...55275"