Create a (semi) automated testing environment
Closed, ResolvedPublic
Actions

Description

Existing setup
We have two test servers that rely on the same database, one of them displaying the state of wikidiff2 BEFORE any of our changes, and the second one with the MOST RECENT version of wikidiff2 (i.e. including our changes).

Task
Create a way we can (semi) automate testing, that ideally shows us

cases where we did not expect a changed diff, but it was changed
cases where we did expect a changed diff, but it was not changed
ideally: cases where we did not receive exactly what we wanted

for specific thresholds.

Details

	Subject	Repo	Branch	Lines +/-
	[WIP] WikiDiff PHPunit Tests	mediawiki/core	master	+2 K -0

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Resolved	• DannyH	T121469 Improve diff compare screen
Resolved	thiemowmde	T139603 Show text changes when moving text chunks (#2)
Resolved	None	T146781 Show changes in moved text chunks (C++)
Resolved	• jkroll	T181494 Investigate what the ideal change detection threshold would be
Resolved	Andrew-WMDE	T183131 Create a (semi) automated testing environment

Event Timeline

Lea_WMDE triaged this task as Medium priority.Dec 18 2017, 11:53 AM

Lea_WMDE created this task.

Lea_WMDE moved this task from Sprint Backlog to Wikidiff2 on the WMDE-QWERTY-XMAS-Sprint-2017-12-20 board.

Maybe something like a simple phpunit test with presets of diffs and expected best results of HTML output.b

Tobi_WMDE_SW added a project: WMDE-QWERTY-Sprint-2018-01-09.Jan 9 2018, 4:32 PM

A rough sketch on what should be done from an IRC conversation this morning

I image something like a unit test that has input ( two texts ) and an expected output ( the marked up diff ).
The marked up div is the ideal we would imagine that looks best to understand what's going on. ( That might be subjective but lets see with our examples )
For the input extract the essential parts from the regression examples we have.
Run wikidiff2 on them and check the output with what we ideally expect it to be.
Then we can play with parameters, run the tests and see which examples work and which don't until we have an ideal setup.

The test should be done as Phpunit MW core tests, so they can be re-used to also test other diff implementations.

Andrew-WMDE claimed this task.Jan 15 2018, 11:50 AM

Change 404294 had a related patch set uploaded (by Andrew-WMDE; owner: Andrew-WMDE):
[mediawiki/core@master] [WIP] WikiDiff PHPunit Tests

https://gerrit.wikimedia.org/r/404294

gerritbot added a project: Patch-For-Review.Jan 15 2018, 1:36 PM

WMDE-Fisch moved this task from Sprint Backlog to Doing on the WMDE-QWERTY-Sprint-2018-01-09 board.Jan 17 2018, 11:15 AM

Lea_WMDE moved this task from Proposed to Currently in sprint on the WMDE-TechWish board.Jan 22 2018, 7:46 PM

WMDE-Fisch closed this task as Resolved.Jan 23 2018, 3:19 PM

WMDE-Fisch moved this task from Doing to Done on the WMDE-QWERTY-Sprint-2018-01-09 board.

WMDE-Fisch moved this task from Currently in sprint to Done on the WMDE-TechWish board.Jan 23 2018, 4:01 PM

Lea_WMDE moved this task from Done to Demoed on the WMDE-TechWish board.Feb 20 2018, 3:31 PM

Change 404294 abandoned by Andrew-WMDE:
[WIP] WikiDiff PHPunit Tests