Page MenuHomePhabricator

Structured unit test results
Closed, DeclinedPublic

Description

Currently the CI jobs are emitting only unstructured unit test results, which are difficult to parse and re-use.

Some CI engines, such as Jenkins, Circle CI and Appveyor understand structured unit tests results.

The two commonly understood structured data formats for unit tests are JUnit XML (and derivatives) and Cucumber JSON.

Travis CI does not plan to implement this in the immediate future. https://github.com/travis-ci/travis-ci/issues/239 https://github.com/travis-ci/travis-ci/issues/2992 https://github.com/travis-ci/travis-ci/issues/3042

However, it is still useful for Jenkins, which is run in Gerrit, and it is useful for local developers and tools to check the results.

pytest and nose appear to have JUnit XML built in : https://pytest.org/latest/usage.html and http://nose.readthedocs.io/en/latest/plugins/xunit.html

Circle CI mentions using https://github.com/django-nose/django-nose for Django.

For Cucumber

This task is to analyse the two formats, verify that the XML outputs of the builtin plugins are correct and usable, and determine if there is a good reason to use Cucumber instead of the built in XML.

Some highly desirable features for the choice between XML or JSON:

Event Timeline

jayvdb created this task.May 20 2016, 3:04 AM
Restricted Application added subscribers: pywikibot-bugs-list, Zppix, Aklapper. · View Herald TranscriptMay 20 2016, 3:04 AM
jayvdb updated the task description. (Show Details)May 22 2016, 9:29 PM
Xqt removed jayvdb as the assignee of this task.Sep 24 2019, 3:39 PM
Xqt lowered the priority of this task from High to Lowest.
Dvorapa moved this task from Backlog to Framework on the Pywikibot-tests board.Apr 3 2020, 4:37 PM
Xqt closed this task as Declined.Sep 22 2020, 8:40 AM