Page MenuHomePhabricator

Parsoid: make roundtrip-test.js run as a Mocha test in CI
Closed, ResolvedPublic

Description

roundtrip-test.js is designed to be run against wiki pages and ensure that a round trip through transform/wikitext/to/html and transform/html/to/wikitext returns the same wikitext again.

We want to run this kind of test in CI as well. The easiest way to achieve this is to make roundtrip-test.js runnable as a Mocha test using the api-testing framework.

Rouch idea for implementation:

  1. Move much of the contents of roundtrip-test.js into a module, say lib/utils/RoundtripTestUtils.js. Keep only the command line interface in roundtrip-test.js, so it works as before. This should be easy, since roundtrip-test.js is already usable as a module.
  2. Create a new Mocha test under tests/api-testing, much like the Parsoid.js suite.
  3. Refactor the runTests() method and create a diffTest() that can be used in a mocha it() block with Chai assertions.
  4. Make a directory tests/roundtrip that contains at least two files with wikitext. Each file will be copied to a wiki page, and the round trip test will be run against it.
  5. To achieve this, loop over the files, and for each file, call it(). Inside the test, copy the contents of the file to a wiki page (see how we do it in before in Parsoid.js), and then call the diffTest() method on that page.

Event Timeline

ssastry renamed this task from Parsoid: make roundtrip-test.js run as a Moch test in CI to Parsoid: make roundtrip-test.js run as a Mocha test in CI.Aug 1 2022, 2:18 PM

Information for Daniel since he asked for this earlier. The round trip testing script runs two kinds of wt2wt tests. One without selser and one with selser. For the non-selser run, it is expected that there will dirty diffs. So, the tests shouldn't assert for absence of diffs. With the dummy comment edit test (adds a comment to the end of the doc) that the script uses, it should be possible to assert that the selser test yields no diffs.

Also, one of the main goals of this mocha tests will primarily be as a smoke test to catch any flagrant breakage to the API or functionality that would cause the tests to fail (http 400, 404, 500, etc.).

So, the tests should be designed with the above in mind.

daniel raised the priority of this task from Medium to High.Sep 5 2022, 9:23 AM

Change 823150 had a related patch set uploaded (by MSantos; author: Derick Alangi):

[mediawiki/services/parsoid@master] api-testing: e2e test for roundtrip-test in CI

https://gerrit.wikimedia.org/r/823150

Change 823150 had a related patch set uploaded (by MSantos; author: Derick Alangi):

[mediawiki/services/parsoid@master] api-testing: e2e test for roundtrip-test in CI

https://gerrit.wikimedia.org/r/823150

Change 823150 merged by jenkins-bot:

[mediawiki/services/parsoid@master] api-testing: e2e test for roundtrip-test in CI

https://gerrit.wikimedia.org/r/823150

Change 848417 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.17.0-a4

https://gerrit.wikimedia.org/r/848417

Change 848417 merged by jenkins-bot:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.17.0-a4

https://gerrit.wikimedia.org/r/848417