Page MenuHomePhabricator

mw.text.unstripNoWiki doesn't unstrip when using Parsoid
Closed, ResolvedPublic

Description

When the output of a module is used in Parsoid, for example in the Visual Editor preview, mw.text.unstripNoWiki doesn't unstrip nowiki tags as it should. This is caused because when using Parsoid, the output type is set to preprocess for the parser, causing it to store the nowiki tags as general tags.

If you have a module (at Module:Test) with the code:

local p = {}

function p.main (frame)
	return mw.text.unstripNoWiki(frame.args.code)
end

return p

And then you call it with:

{{#invoke:Test|main|code=<nowiki>[[Link]]</nowiki>}}

The parser will return Link whereas Parsoid will return [[Link]]. Loading the page with ?action=raw&templates=expand returns <nowiki>[[Link]]</nowiki> when the expected output would be [[Link]] as the nowiki tags should have been stripped in the module.

Looking at the stripstate, when using the regular parser it has the data

Array
(
    [nowiki] => Array
        (
            [-nowiki-00000000] => [[Link]]
        )

    [general] => Array
        (
        )

)

But when using Parsoid or ?action=raw&templates=expand it is

Array
(
    [nowiki] => Array
        (
        )

    [general] => Array
        (
            [-nowiki-00000000] => <nowiki>[[Link]]</nowiki>
        )

)

Event Timeline

ssastry subscribed.

T203293 *was* a symptom of this underlying issue and in that case, they were using nowiki as a hack. But, there might be other legitimate uses of using nowiki in transclusion arguments which modules might want to strip out to process the protected text.

See the discussion in T203293 for why this fails in Parsoid right now. But, it is the same reason as with T199852: mw.text.unstripNoWiki doesn't work in special:ExpandTemplates. However, unlike Special:ExpandTemplates, till T114432: [RFC] Heredoc arguments for templates (aka "hygienic" or "long" arguments) is implemented, this is going to be an issue on pages where nowikis are used to protect all or pieces of a template argument as in the example from T258955.

Change 822177 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/core@master] Add functionality to enable Scribunto to support Parsoid properly

https://gerrit.wikimedia.org/r/822177

Change 822178 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/extensions/Scribunto@master] Provide modules access to nowiki content when Parsoid is involved

https://gerrit.wikimedia.org/r/822178

ssastry triaged this task as Medium priority.

Change 822447 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/extensions/Scribunto@master] Enable testing with Parsoid + add failing test for T272507

https://gerrit.wikimedia.org/r/822447

Change 822447 merged by jenkins-bot:

[mediawiki/extensions/Scribunto@master] Enable testing with Parsoid + add failing test for T272507

https://gerrit.wikimedia.org/r/822447

Change 826409 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/extensions/Scribunto@master] Add Parsoid-specific support for handling preprocessed nowikis

https://gerrit.wikimedia.org/r/826409

Change 826408 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/core@master] Add support to enable Scribunto & Parsoid to handle nowikis properly

https://gerrit.wikimedia.org/r/826408

Change 826408 merged by jenkins-bot:

[mediawiki/core@master] Add support to enable Scribunto & Parsoid to handle nowikis properly

https://gerrit.wikimedia.org/r/826408

Change 822177 abandoned by Subramanya Sastry:

[mediawiki/core@master] Add functionality to enable Scribunto to support Parsoid properly

Reason:

In favour of Ied0295feab06027a8df885b3215435e596f0353b

https://gerrit.wikimedia.org/r/822177

Change 822178 abandoned by Subramanya Sastry:

[mediawiki/extensions/Scribunto@master] Provide modules access to nowiki content when Parsoid is involved

Reason:

In favour of I1613ac7bd60cf3ef4a3308b08ea3705b3cf2dee0

https://gerrit.wikimedia.org/r/822178

Change 826409 merged by jenkins-bot:

[mediawiki/extensions/Scribunto@master] Add Parsoid-specific support for handling preprocessed nowikis

https://gerrit.wikimedia.org/r/826409

Change #1127162 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/extensions/Scribunto@master] Update output of unstripNoWiki parser test

https://gerrit.wikimedia.org/r/1127162

Change #1123779 had a related patch set uploaded (by C. Scott Ananian; author: Subramanya Sastry):

[mediawiki/core@master] Process strip markers recursively in split

https://gerrit.wikimedia.org/r/1123779

Change #1123779 merged by jenkins-bot:

[mediawiki/core@master] Fixes to "Parsoid Fragment Support v2"

https://gerrit.wikimedia.org/r/1123779

Change #1127162 merged by C. Scott Ananian:

[mediawiki/extensions/Scribunto@master] Update output of unstripNoWiki parser test

https://gerrit.wikimedia.org/r/1127162

Change #1127076 had a related patch set uploaded (by Jforrester; author: Subramanya Sastry):

[mediawiki/core@wmf/1.44.0-wmf.20] Fixes to "Parsoid Fragment Support v2"

https://gerrit.wikimedia.org/r/1127076

Change #1127076 abandoned by Subramanya Sastry:

[mediawiki/core@wmf/1.44.0-wmf.20] Fixes to "Parsoid Fragment Support v2"

Reason:

We won't backport this. https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1127630 is also needed. So, we will just let them both ride the train. Then on Thursday, run rt-testing in Parsoid with this enabled, and then turn on v2 fragment mode the following week. So, we'll lose a week, but this has given us enough trouble that it is better to do a bit more thorough testing.

https://gerrit.wikimedia.org/r/1127076