Weekly report 10: Testing framework for GRASS GIS
2014-07-26 04:35:45 GMT
I significantly improved the test file page which is the central page of the report in the sense that there you can get all the details. I also added details about successful and failed (individual) tests to main index page (the one with links to all test files). Additionally, I created different set of pages which sorts the test files from the point of view of testsuite directories which might be the primary way of browsing tests since it is aligned with how tests are included in the source code. The page for a given testsuite also contains approximate information about authors of tests and authors of tested code.
Each test file has now a key-value file which contains information about success, number of tests, and other information. These files can and are collected to obtain summaries. The key-value format was selected mainly because of its usage in GRASS GIS (otherwise, for example Python documentation for inputs and outputs recommends JSON ).
I believe that I designed report classes in the way that they can be extended but already I can see some room for refactoring or at least for better organization. This might be a topic for week 13.
I did not created any XML output. The situation seems unclear and a lot of transformations are done to get XML working with different tools. On the other hand, with some tools even trivial XML with numbers (test statistics) should work and this should be easy to create from key-value files or directly. Also there is a unittest-xml-reporting package  for Python `unittest` which should create XML in the style of Ant/JUnit. In theory this should work directly with `gunittest`. I was not considering database output much but I think that it is not needed now and it can be created from key-value files if necessary.
A also did not focus on CSS of the HTML reports. They are readable but that's all. Some CSS might be added in the future.
There is currently 20 successfully running test files and 5 failing test files, 9 successfully running testsuites and 4 failing testsuites, and 686 successfully running tests and 7 failing tests (533 of the tests are from PyGRASS module test).
I also added some tests for `r.mapcalc` `rand()` function and new behavior with seed in r61355.
# open file "testreport/index.html" or "testreport/testsuites.html"
I plan to rewrite some of the existing tests using `gunittest` or write new ones. I already started to write something for `r.slope.aspect` and `r.mapcalc`.
If time allows, I would like to add some key-value summary files to reports, include some test metadata such as GRASS location or architecture, and also introduce more options to report generation (currently, it cannot be configured).
Additionally, It would be nice to have all path in files shortened so that they don't include `dist...` or full path to the file and also it would be nice to have an example of running the tests automatically in some trivial form.
3. Are you blocked on anything?
Wiki with screenshots:
_______________________________________________ grass-dev mailing list grass-dev <at> lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev