Page MenuHomePhabricator

Create a (semi) automated testing environment
Closed, ResolvedPublic

Description

Existing setup
We have two test servers that rely on the same database, one of them displaying the state of wikidiff2 BEFORE any of our changes, and the second one with the MOST RECENT version of wikidiff2 (i.e. including our changes).

Task
Create a way we can (semi) automate testing, that ideally shows us

  • cases where we did not expect a changed diff, but it was changed
  • cases where we did expect a changed diff, but it was not changed
  • ideally: cases where we did not receive exactly what we wanted

for specific thresholds.

Details

Related Gerrit Patches:
mediawiki/core : master[WIP] WikiDiff PHPunit Tests

Event Timeline

Lea_WMDE triaged this task as Normal priority.Dec 18 2017, 11:53 AM
Lea_WMDE created this task.

Maybe something like a simple phpunit test with presets of diffs and expected best results of HTML output.b

A rough sketch on what should be done from an IRC conversation this morning

  • I image something like a unit test that has input ( two texts ) and an expected output ( the marked up diff ).
  • The marked up div is the ideal we would imagine that looks best to understand what's going on. ( That might be subjective but lets see with our examples )
  • For the input extract the essential parts from the regression examples we have.
  • Run wikidiff2 on them and check the output with what we ideally expect it to be.
  • Then we can play with parameters, run the tests and see which examples work and which don't until we have an ideal setup.

The test should be done as Phpunit MW core tests, so they can be re-used to also test other diff implementations.

Change 404294 had a related patch set uploaded (by Andrew-WMDE; owner: Andrew-WMDE):
[mediawiki/core@master] [WIP] WikiDiff PHPunit Tests

https://gerrit.wikimedia.org/r/404294

WMDE-Fisch closed this task as Resolved.Jan 23 2018, 3:19 PM
WMDE-Fisch moved this task from Doing to Done on the WMDE-QWERTY-Sprint-2018-01-09 board.
Lea_WMDE moved this task from Done to Demoed on the WMDE-QWERTY-Team board.Feb 20 2018, 3:31 PM

Change 404294 abandoned by Andrew-WMDE:
[WIP] WikiDiff PHPunit Tests

Reason:
No longer needed

https://gerrit.wikimedia.org/r/404294