Page MenuHomePhabricator

Parsoid does not correctly render <noinclude> if used with templates
Closed, ResolvedPublicBUG REPORT

Description

Steps to replicate the issue (include links if applicable):

* Example template call:
<noinclude>{{NewsList
| Month=01
| Year=2023
| Image=[[File:Pernegg Kloster Turm 20170128.jpg|250px|links|Eckturm der Umfassungsmauern des ehemaligen Klosters Pernegg, Österreich]]
| 1=
</noinclude>
<!-- ... -->

What happens?:

  • With legacy parser all is OK
  • With parsoid, <noinclude> is handled like <nowiki>: the wiki syntax of the template is shown. But in this case, <noinclude> should be ignored.

What should have happened instead?:

  • The Wiki syntax should not be shown. Instead the template should be correctly parsed

Software version (on Special:Version page; skip for WMF-hosted wikis like Wikipedia):

Other information (browser name/version, screenshots, etc.):

  • All browser, desktop

Event Timeline

How common is this pattern where wikitext strings are stictched together across <noinclude> / <includeonly> blocks? Parsoid currently doesn't support it (by design) - it treats contents of <noinclude> / <includeonly> blocks are self-contained. We can revisit that design decision if really necessary -- but before we take that on, we would like to understand if this pattern is specific to a few pages on a this wiki and if there are alternative wikitext formulations that can work instead.

The use of templates is a Mediawiki core functionality. It should properly work in any case both in main, user and template namespaces. In the task described here it is a very easy case because <noinclude> can be ignored. This should be an easy task for programmers.

In case of big wikis like English Wikipedia it seems not possible to detect how common/often this (official) feature was used. But all authors will expect that the new parser should work in a similar manner as it was done by the legacy parser. It think it is better to revisit an unfortunate design decision than to upset authors and force them to generate alternatives.

We understand the need to replicate legacy parser behavior and that it should work in all namespaces. Parsoid's design is different because it has many other requirements besides rendering a page: support HTML editing without dirty diffs, supporting richer markup for tools, and more recently, we have been exploring T363421: Prototype selective HTML updates in Parsoid which can lead to pages being updated upto 5x faster when a dependent template is edited. So, given the overall requirements, we have made design decisions in the past to enable us to meet those goals. I say all that as a preamble to indicate that there are tradeoffs and if a particular wikitext use case is not used often, it is not unreasonable to look for alternatives.

In this specific case, it is not a simple matter of removing the "<noinclude>" tags and processing the wikitext. If it were, we would have done it. If we remove the noinclude tags and leave behind no information about them, on visual editor edits, that page will get a dirty diff. Additionally, if we process the wikitext after stripping the tags, the string offsets generated for all the generated HTML will no longer match up with the original wikitext which has a more serious dirty diff impact on any edit to the page. So, it is more complicated than it appears for these reasons.

It is not unsolveable and we can explore strategies, but that is what I meant above by tradeoffs. In evaluating these tradeoffs, it is useful to understand how commonly this pattern is used. I hope this helps clarify my original question.

Another example from https://en.wikipedia.org/wiki/Wikipedia:Arbitration/Requests/Case?useparsoid=1

{{Wikipedia:Arbitration/Requests/Case/Header<noinclude>|width=auto</noinclude>}}
ABreault-WMF triaged this task as High priority.
ABreault-WMF moved this task from Backlog to In Progress on the Content-Transform-Team-WIP board.

Change #1090576 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/services/parsoid@master] [WIP] String gluing across include directive boundary

https://gerrit.wikimedia.org/r/1090576

Change #1090576 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/services/parsoid@master] String gluing across include directive boundary

https://gerrit.wikimedia.org/r/1090576

Change #1090576 merged by jenkins-bot:

[mediawiki/services/parsoid@master] String gluing across include directive boundary

https://gerrit.wikimedia.org/r/1090576

Change #1092952 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.21.0-a7

https://gerrit.wikimedia.org/r/1092952

Change #1092952 merged by jenkins-bot:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.21.0-a7

https://gerrit.wikimedia.org/r/1092952

Change #1093356 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/vendor@wmf/1.44.0-wmf.4] Bump wikimedia/parsoid to 0.21.0-a7

https://gerrit.wikimedia.org/r/1093356

Change #1093358 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/vendor@wmf/1.44.0-wmf.4] Bump wikimedia/parsoid to 0.21.0-a7

https://gerrit.wikimedia.org/r/1093358

Change #1093356 abandoned by Arlolra:

[mediawiki/vendor@wmf/1.44.0-wmf.4] Bump wikimedia/parsoid to 0.21.0-a7

https://gerrit.wikimedia.org/r/1093356

Change #1093358 merged by jenkins-bot:

[mediawiki/vendor@wmf/1.44.0-wmf.4] Bump wikimedia/parsoid to 0.21.0-a7

https://gerrit.wikimedia.org/r/1093358

Mentioned in SAL (#wikimedia-operations) [2024-11-20T22:03:03Z] <cjming@deploy2002> Started scap sync-world: Backport for [[gerrit:1093358|Bump wikimedia/parsoid to 0.21.0-a7 (T373776 T380333)]], [[gerrit:1093359|Bump wikimedia/parsoid to 0.21.0-a7 (T380333)]]

Mentioned in SAL (#wikimedia-operations) [2024-11-20T22:08:46Z] <cjming@deploy2002> arlolra, cjming: Backport for [[gerrit:1093358|Bump wikimedia/parsoid to 0.21.0-a7 (T373776 T380333)]], [[gerrit:1093359|Bump wikimedia/parsoid to 0.21.0-a7 (T380333)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2024-11-20T22:20:14Z] <cjming@deploy2002> Finished scap sync-world: Backport for [[gerrit:1093358|Bump wikimedia/parsoid to 0.21.0-a7 (T373776 T380333)]], [[gerrit:1093359|Bump wikimedia/parsoid to 0.21.0-a7 (T380333)]] (duration: 17m 11s)