Page MenuHomePhabricator

Preprocessor should handle -{...}- variant constructs in template arguments
Closed, ResolvedPublic

Description

Because the preprocessor doesn't know about language converter markup:

{{echo|-{R|foo}-}}

is parsed incorrectly -- the preprocessor seems this as two arguments, -{R and foo}-.

Related Objects

StatusSubtypeAssignedTask
OpenNone
OpenNone
OpenNone
StalledNone
OpenNone
StalledNone
Resolvedovasileva
OpenNone
DuplicateNone
OpenNone
InvalidNone
ResolvedPchelolo
Resolvedmobrovac
ResolvedPchelolo
ResolvedJdforrester-WMF
ResolvedMarkTraceur
ResolvedJdforrester-WMF
Resolvedcscott
OpenNone
StalledNone
OpenNone
OpenNone
OpenNone
Opencscott
OpenNone
Opencscott
Invalid GWicke
Resolvedliangent
Resolvedthiemowmde
OpenNone
Resolvedcscott
Resolvedcscott
ResolvedElitre
Resolvedcscott
Opencscott
Opencscott
Opencscott
OpenNone

Event Timeline

cscott created this task.Sep 21 2016, 6:08 PM
Restricted Application removed a project: Patch-For-Review. · View Herald TranscriptSep 21 2016, 6:08 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Change 311849 had a related patch set uploaded (by C. Scott Ananian):
Protect language converter markup in the preprocessor.

https://gerrit.wikimedia.org/r/311849

Change 311849 merged by jenkins-bot:
Protect language converter markup in the preprocessor.

https://gerrit.wikimedia.org/r/311849

cscott added a comment.EditedMar 30 2017, 8:20 PM

A patch for this was merged (https://gerrit.wikimedia.org/r/311849), but it was then reverted (turned off, really) due to unexpected interactions with -{{ and similar constructs (see https://gerrit.wikimedia.org/r/330315 and https://gerrit.wikimedia.org/r/330323). This was T153761.

A second version of the patch looks to be more successful: https://gerrit.wikimedia.org/r/333997

However, there are about 2,000 articles on the various big wikis which require some fixup before we can merge this. See https://www.mediawiki.org/wiki/Parsoid/Language_conversion/Preprocessor_fixups for details.

Change 333997 had a related patch set uploaded (by C. Scott Ananian):
[mediawiki/core@master] Protect language converter markup in the preprocessor (take 2).

https://gerrit.wikimedia.org/r/333997

Amire80 added a subscriber: Amire80.May 8 2017, 7:45 AM

However, there are about 2,000 articles on the various big wikis which require some fixup before we can merge this. See https://www.mediawiki.org/wiki/Parsoid/Language_conversion/Preprocessor_fixups for details.

How about the less big wikis?

cscott added a comment.May 8 2017, 9:44 PM

Sure. As discussed here https://www.mediawiki.org/wiki/Topic:Tq16054ko2jvisgw the process is mostly automated, I just need to download dumps locally and re-run the grep. (The process is described at https://www.mediawiki.org/wiki/Parsoid/Language_conversion/Preprocessor_fixups#How_do_we_tell_how_widespread_this_is.3F if you feel impatient.)

It might also be possible to add a Parsoid linter rule and/or a maintenance category in the PHP parser for "use of language converter markup w/o LanguageConverter enabled", but both of those solutions would require that the relevant patches be merged & deployed first -- and we're trying to fix up as much as possible before that happens.

Change 333997 merged by jenkins-bot:
[mediawiki/core@master] Protect language converter markup in the preprocessor (take 2).

https://gerrit.wikimedia.org/r/333997

cscott closed this task as Resolved.Jun 5 2017, 2:10 AM
cscott claimed this task.

Deployed in 1.30.0 wmf.2