Fix selser result pass/failure detection in parserTests.js
Open, LowPublic

Description

When selser runs (html2wt) on modified HTML, we have to figure out whether the serialized wikitext is correct or not. However, there is no original wikitext to compare against since the DOM has been modified.

Currently, parserTests runs wt2wt on the same modified HTML and compares selser output with wt2wt output. wt2wt uses the regular full serializer. This pass/failure detection implicitly assumes that the output from wt2wt is correct. However, this is flawed -- it might actually be the case that selser output is correct and wt2wt is incorrect since wt2wt might be normalizing wikitext whereas selser wont (which is the whole point of selser, so to speak). This was not as much of an issue till a while back since wt2wt was normalizing less and parserTests used to run in rt-testing mode which used src-based rting for templates and the like. However, this is becoming more of a problem as we move towards greater normalization (ex: bug 51004), and when the regular serializer no longer run in rt-testing mode.

So, we need to figure out a more robust solution to this problem.


Version: unspecified
Severity: normal

Details

Security
None
Reference
bz51718
bzimport set Reference to bz51718.
ssastry created this task.Jul 19 2013, 10:53 PM

Change 85952 had a related patch set uploaded by Arlolra:
WIP: Remember the output of failing (blacklisted) tests

https://gerrit.wikimedia.org/r/85952

Change 85952 merged by jenkins-bot:
Remember the output of failing (blacklisted) tests

https://gerrit.wikimedia.org/r/85952

Subbu / Gabriel: Patch was merged month ago - is there more work left here, or can you close this ticket as RESOLVED FIXED?

We merged a different workable solution for now, but I still keep hoping for a smarter solution to turn up. Leaving this here in case someone has a smart idea for this when they are bored and idle and thinking about this. ;-)

weird. why is marcoil suddenly the default?

I think I'm the default for Parsoid/tests :)

Arlolra removed GWicke as the assignee of this task.Nov 25 2014, 6:57 PM
Arlolra added a project: Parsoid.
Arlolra set Security to None.
ssastry moved this task from Backlog to Testing on the Parsoid board.Dec 20 2014, 4:15 AM
ssastry lowered the priority of this task from "Normal" to "Low".Feb 2 2015, 11:31 PM
marcoil moved this task from Testing to Backlog on the Parsoid board.Feb 13 2015, 12:51 PM

Add Comment