Currently the CI jobs are emitting only unstructured unit test results, which are difficult to parse and re-use.
Some CI engines, such as Jenkins, Circle CI and Appveyor understand structured unit tests results.
Travis CI does not plan to implement this in the immediate future. https://github.com/travis-ci/travis-ci/issues/239 https://github.com/travis-ci/travis-ci/issues/2992 https://github.com/travis-ci/travis-ci/issues/3042
However, it is still useful for Jenkins, which is run in Gerrit, and it is useful for local developers and tools to check the results.
pytest and nose appear to have JUnit XML built in : https://pytest.org/latest/usage.html and http://nose.readthedocs.io/en/latest/plugins/xunit.html
Circle CI mentions using https://github.com/django-nose/django-nose for Django.
- https://pypi.python.org/pypi/pytest-json (is it Cucumber compatible?)
- https://pypi.python.org/pypi/pytest-bdd has a compatible output ( https://github.com/pytest-dev/pytest-bdd/blob/master/pytest_bdd/cucumber_json.py )
This task is to analyse the two formats, verify that the XML outputs of the builtin plugins are correct and usable, and determine if there is a good reason to use Cucumber instead of the built in XML.
Some highly desirable features for the choice between XML or JSON:
- Jenkins support
- a tool that already exists to check which tests error/fail/skip/expected failure, which can be used to solve T135795: CI expected failures and skips.