PHP-parser + Remex combo output differs from PHP-parser + Tidy combo on some dl-dt wikitext snippets
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	ssastry
	Sep 5 2017, 11:46 PM

Description

See https://www.mediawiki.org/wiki/User:SSastry_(WMF)/Tests/Remex?action=parsermigration-edit

Parsoid: https://www.mediawiki.org/api/rest_v1/page/html/User:SSastry_(WMF)%2FTests%2FRemex

Details

	Subject	Repo	Branch	Lines +/-
	Fix bug in dl-dt list output generation	mediawiki/core	master	+22 -2

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Resolved	Arlolra	T109897 Table parsing diffs: Parsoid adds implicit <td>s after a \|- if explicit pipe is not present
Resolved	Arlolra	T109650 Minor P-wrapping diff between Parsoid & PHP Parser+Tidy combo
Resolved	Arlolra	T110004 DOM Pass for wrapping bare text found in <body> and other "block" (in html4-parlance) nodes like <blockquote>, <td>, <th>.
Declined	None	T65699 Parsoid and Tidy differ in how they deal with misnested tags
Declined	ssastry	T69452 Empty elements in DOM: PHP parser+tidy strips them; Parsoid doesn't
Resolved	ssastry	T175706 Progressively switch Wikimedia wikis from Tidy to RemexHTML
Open	None	T49544 <references/> list item must not wrap the text in <span>
Resolved	• tstarling	T89331 Replace HTML4 Tidy in MW parser with an equivalent HTML5 based tool
Resolved	ssastry	T175099 PHP-parser + Remex combo output differs from PHP-parser + Tidy combo on some dl-dt wikitext snippets

Event Timeline

ssastry created this task.Sep 5 2017, 11:46 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 5 2017, 11:46 PM

ssastry triaged this task as Medium priority.Sep 5 2017, 11:46 PM

• tstarling added a project: RemexHtml.Sep 5 2017, 11:49 PM

See transcript below. This is not a Remex bug after all. Tidy does overeager fixup of the HTML that is not mandated by the HTML5 spec. The reason Parsoid doesn't have the error is because it closes the </dt> tag and opens a <dl> when it encounters a newline, which in turns mimics PHP-parser +Tidy behavior. So, this is more like a PHP parser bug. This will need a fix to the PHP parser output to close the </dt> tag and open a <dl> tag like Parsoid does.

----- PARSOID -----
[subbu@earth maintenance] echo ';a\n:*b' | parse.js --trace html --normalize
0-[HTML]       | {"type":"TagTk","name":"dl","attribs":[],"dataAttribs":{"tsr":[0,0],"tmp":{"tagId":1}}}
0-[HTML]       | {"type":"TagTk","name":"dt","attribs":[],"dataAttribs":{"tsr":[0,1],"tmp":{"tagId":2}}}
0-[HTML]       | "a"
0-[HTML]       | {"type":"EndTagTk","name":"dt","attribs":[],"dataAttribs":{"tmp":{}}}
0-[HTML]       | {"type":"NlTk","dataAttribs":{"tsr":[2,3],"tmp":{}}}
0-[HTML]       | {"type":"TagTk","name":"dd","attribs":[],"dataAttribs":{"tsr":[3,4],"tmp":{"tagId":3}}}
0-[HTML]       | {"type":"TagTk","name":"ul","attribs":[],"dataAttribs":{"tsr":[4,4],"tmp":{"tagId":4}}}
0-[HTML]       | {"type":"TagTk","name":"li","attribs":[],"dataAttribs":{"tsr":[4,5],"tmp":{"tagId":5}}}
0-[HTML]       | "b"
0-[HTML]       | {"type":"EndTagTk","name":"li","attribs":[],"dataAttribs":{"tmp":{}}}
0-[HTML]       | {"type":"EndTagTk","name":"ul","attribs":[],"dataAttribs":{"tmp":{}}}
0-[HTML]       | {"type":"EndTagTk","name":"dd","attribs":[],"dataAttribs":{"tmp":{}}}
0-[HTML]       | {"type":"EndTagTk","name":"dl","attribs":[],"dataAttribs":{"tmp":{}}}
0-[HTML]       | {"type":"NlTk","dataAttribs":{"tsr":[6,7],"tmp":{}}}
0-[HTML]       | {"type":"EOFTk"}

<dl>
<dt>a</dt>
<dd>
<ul>
<li>b</li>
</ul>
</dd>
</dl>

----- PHP PARSER WITHOUT TIDY -----
[subbu@earth maintenance] echo ';a\n:*b' | php parse.php 
<div class="mw-parser-output"><dl><dt>a
<ul><li>b</li></ul></dt></dl>
</div>

----- PHP PARSER WITH TIDY -----
[subbu@earth maintenance] echo ';a\n:*b' | php parse.php --tidy
<div class="mw-parser-output"><dl>
<dt>a</dt>
<dd>
<ul>
<li>b</li>
</ul>
</dd>
</dl>

</div>

----- PARSOID when run on PHP PARSER's output -----
[subbu@earth maintenance] echo '<dl><dt>a\n<ul><li>b</li></ul></dt></dl>' | parse.js --normalize
<dl>
<dt>a
<ul>
<li>b</li>
</ul>
</dt>
</dl>

ssastry renamed this task from Remex HTML5 tree building bug in dl-dt handling? to PHP-parser + Remex combo output differs from PHP-parser + Tidy combo on some dl-dt wikitext snippets.Sep 6 2017, 10:17 PM

ssastry updated the task description. (Show Details)

Change 376446 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[mediawiki/core@master] WIP: Fix bug in dl-dt list output generation

https://gerrit.wikimedia.org/r/376446

gerritbot added a project: Patch-For-Review.Sep 6 2017, 10:49 PM

ssastry added a project: MediaWiki-Parser.Sep 7 2017, 4:17 AM

Change 376446 merged by jenkins-bot:
[mediawiki/core@master] Fix bug in dl-dt list output generation

https://gerrit.wikimedia.org/r/376446

ssastry closed this task as Resolved.Sep 14 2017, 3:54 AM

ssastry claimed this task.

ssastry added a subtask: T175095: Enable RemexHTML on mediawiki.org and testwiki.Sep 14 2017, 3:56 AM

ssastry removed a subtask: T175095: Enable RemexHTML on mediawiki.org and testwiki.Sep 14 2017, 7:09 PM

ssastry added a parent task: T175095: Enable RemexHTML on mediawiki.org and testwiki.

Jdforrester-WMF edited parent tasks, added: T89331: Replace HTML4 Tidy in MW parser with an equivalent HTML5 based tool; removed: T175095: Enable RemexHTML on mediawiki.org and testwiki.Oct 11 2017, 11:30 PM

Aklapper removed a subscriber: Parsing-Team--ARCHIVED.May 24 2023, 1:09 PM