Jenkins integration would make the detection of regressions more reliable and provide us with a publicly accessible record of test results.
For this, we need to provide a test output format from parserTests that JUnit understands. The PHP parser test runner supports 'Test Results (XML)' as described in http://www.phpunit.de/manual/3.6/en/logging.html and http://windyroad.org/dl/Open%20Source/JUnit.xsd.
Version: unspecified
Severity: enhancement
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | None | T42933 Automated testing in Jenkins - Tracking | |||
| Resolved | ssastry | T40092 Parsoid / parserTests: Add JUnit XML test output for Jenkins integration |
But, we don't appear to be using anything to run tests right now. So we would either need to start using a testing framework (maybe making the test code easier to read) or build out an XML-generator library in the tests/ directory.
Subbu has started to use Jasmine for some purely-JS tests. We could hook up Jasmine to run all test cases in parserTests.txt in all possible directions (wikitext -> html, html -> wikitext, wikitext -> wikitext rt, html -> html rt). That should give us XML output as described above.
Given that our focus will switch away from the JS implementation soonish, it might not be worth to spend too much time on this right now. More tests in a language-indenpendent format is a higher priority in any case.
This won't be an issue anymore when the tests are run from MediaWiki built-in hooks (either in core or as an extension). Since PHPUnit already supports it and Jenkins is already using it properly that way as well.