Fix selser result pass/failure detection in parserTests.js


When selser runs (html2wt) on modified HTML, we have to figure out whether the serialized wikitext is correct or not. However, there is no original wikitext to compare against since the DOM has been modified.

Currently, parserTests runs wt2wt on the same modified HTML and compares selser output with wt2wt output. wt2wt uses the regular full serializer. This pass/failure detection implicitly assumes that the output from wt2wt is correct. However, this is flawed -- it might actually be the case that selser output is correct and wt2wt is incorrect since wt2wt might be normalizing wikitext whereas selser wont (which is the whole point of selser, so to speak). This was not as much of an issue till a while back since wt2wt was normalizing less and parserTests used to run in rt-testing mode which used src-based rting for templates and the like. However, this is becoming more of a problem as we move towards greater normalization (ex: bug 51004), and when the regular serializer no longer run in rt-testing mode.

So, we need to figure out a more robust solution to this problem.

Version: unspecified
Severity: normal

bzimport added a project: Parsoid-Tests.Via ConduitNov 22 2014, 1:52 AM
bzimport set Reference to bz51718.
ssastry created this task.Via LegacyJul 19 2013, 10:53 PM
gerritbot added a comment.Via ConduitSep 25 2013, 4:00 AM

Change 85952 had a related patch set uploaded by Arlolra:
WIP: Remember the output of failing (blacklisted) tests

gerritbot added a comment.Via ConduitOct 1 2013, 10:33 PM

Change 85952 merged by jenkins-bot:
Remember the output of failing (blacklisted) tests

Aklapper added a comment.Via ConduitFeb 12 2014, 3:58 PM

Subbu / Gabriel: Patch was merged month ago - is there more work left here, or can you close this ticket as RESOLVED FIXED?

ssastry added a comment.Via ConduitFeb 12 2014, 6:39 PM

We merged a different workable solution for now, but I still keep hoping for a smarter solution to turn up. Leaving this here in case someone has a smart idea for this when they are bored and idle and thinking about this. ;-)

Arlolra added a comment.Via ConduitFeb 12 2014, 7:48 PM

weird. why is marcoil suddenly the default?

marcoil added a comment.Via ConduitFeb 13 2014, 10:50 AM

I think I'm the default for Parsoid/tests :)

Arlolra placed this task up for grabs.Via WebNov 25 2014, 6:57 PM
Arlolra added a project: Parsoid.
Arlolra set Security to None.
ssastry moved this task to Testing on the Parsoid workboard.Via WebDec 20 2014, 4:15 AM
ssastry lowered the priority of this task from "Normal" to "Low".Via WebFeb 2 2015, 11:31 PM
marcoil moved this task to Backlog on the Parsoid workboard.Via WebFeb 13 2015, 12:51 PM

Add Comment