Preprocessor should handle -{...}- variant constructs in template arguments
Closed, ResolvedPublic

Description

Because the preprocessor doesn't know about language converter markup:

{{echo|-{R|foo}-}}

is parsed incorrectly -- the preprocessor seems this as two arguments, -{R and foo}-.

cscott created this task.Sep 21 2016, 6:08 PM
Restricted Application removed a project: Patch-For-Review. · View Herald TranscriptSep 21 2016, 6:08 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Change 311849 had a related patch set uploaded (by C. Scott Ananian):
Protect language converter markup in the preprocessor.

https://gerrit.wikimedia.org/r/311849

Change 311849 merged by jenkins-bot:
Protect language converter markup in the preprocessor.

https://gerrit.wikimedia.org/r/311849

cscott added a comment.EditedMar 30 2017, 8:20 PM

A patch for this was merged (https://gerrit.wikimedia.org/r/311849), but it was then reverted (turned off, really) due to unexpected interactions with -{{ and similar constructs (see https://gerrit.wikimedia.org/r/330315 and https://gerrit.wikimedia.org/r/330323). This was T153761.

A second version of the patch looks to be more successful: https://gerrit.wikimedia.org/r/333997

However, there are about 2,000 articles on the various big wikis which require some fixup before we can merge this. See https://www.mediawiki.org/wiki/Parsoid/Language_conversion/Preprocessor_fixups for details.

Change 333997 had a related patch set uploaded (by C. Scott Ananian):
[mediawiki/core@master] Protect language converter markup in the preprocessor (take 2).

https://gerrit.wikimedia.org/r/333997

Amire80 added a subscriber: Amire80.May 8 2017, 7:45 AM

However, there are about 2,000 articles on the various big wikis which require some fixup before we can merge this. See https://www.mediawiki.org/wiki/Parsoid/Language_conversion/Preprocessor_fixups for details.

How about the less big wikis?

cscott added a comment.May 8 2017, 9:44 PM

Sure. As discussed here https://www.mediawiki.org/wiki/Topic:Tq16054ko2jvisgw the process is mostly automated, I just need to download dumps locally and re-run the grep. (The process is described at https://www.mediawiki.org/wiki/Parsoid/Language_conversion/Preprocessor_fixups#How_do_we_tell_how_widespread_this_is.3F if you feel impatient.)

It might also be possible to add a Parsoid linter rule and/or a maintenance category in the PHP parser for "use of language converter markup w/o LanguageConverter enabled", but both of those solutions would require that the relevant patches be merged & deployed first -- and we're trying to fix up as much as possible before that happens.

Change 333997 merged by jenkins-bot:
[mediawiki/core@master] Protect language converter markup in the preprocessor (take 2).

https://gerrit.wikimedia.org/r/333997

cscott closed this task as Resolved.Jun 5 2017, 2:10 AM
cscott claimed this task.

Deployed in 1.30.0 wmf.2