Support for parser functions using "isHTML"?
Open, MediumPublic
Actions

Assigned To

None

Authored By

	Arlolra
	Apr 1 2021, 6:22 PM

Description

Esoteric flags like "isHTML" don't fly with Parsoid

https://github.com/wikimedia/mediawiki/blob/master/includes/parser/Parser.php#L4895

Currently, Parsoid expects parser functions to return wikitext during preprocessing. However, the presence of this "isHTML" flag means that some parser functions return html that the legacy parser armours for them. That's what is happening in T257344

Maybe they should do something that's more aware of the $parser->getOutputType()? That's what the patch in T257344 tries to do so that only wikitext is return when preprocessing.

Something to be aware of when designing Parsoid's setFunctionHook in T268144

Details

	Subject	Repo	Branch	Lines +/-
	[DNM]: Use special strip markers for HTML from parser functions	mediawiki/core	master	+4 -3
	Return an extension tag when preprocessing a parser function	mediawiki/extensions/CategoryTree	master	+8 -3

Customize query in gerrit

Related Objects

Mentioned In: T318060: Scary transcluding a Wikipedia page with inline math renders that inline math incorrectly
T178588: Handle parser functions returning raw html
rEPHN26cb40e430bf: Use tag instead of a parser function
T317112: Phonos does not display correctly in Visual Editor
T289545: Parsoid doesn't respect strip state markers found in preprocessor output
Mentioned Here: T257344: The #categorytree magic word doesn't seem to work in VisualEditor
T268144: Add setFunctionHook equivalent support to Parsoid Extension API

Event Timeline

Arlolra created this task.Apr 1 2021, 6:22 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 1 2021, 6:22 PM

Arlolra triaged this task as Medium priority.Apr 1 2021, 6:23 PM

Arlolra moved this task from Needs Triage to Missing Functionality on the Parsoid board.

Change 676436 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/extensions/CategoryTree@master] Help Parsoid out by returning an ext tag when preprocessing

https://gerrit.wikimedia.org/r/676436

gerritbot added a project: Patch-For-Review.Apr 1 2021, 6:25 PM

Change 676436 merged by jenkins-bot:

[mediawiki/extensions/CategoryTree@master] Return an extension tag when preprocessing a parser function

https://gerrit.wikimedia.org/r/676436

ReleaseTaggerBot added a project: MW-1.36-notes (1.36.0-wmf.38; 2021-04-06).Apr 4 2021, 3:00 PM

Maintenance_bot removed a project: Patch-For-Review.Apr 4 2021, 3:10 PM

Arlolra mentioned this in T289545: Parsoid doesn't respect strip state markers found in preprocessor output.Aug 24 2021, 2:42 PM

matmarex mentioned this in T317112: Phonos does not display correctly in Visual Editor.Sep 6 2022, 10:50 PM

MusikAnimal mentioned this in rEPHN26cb40e430bf: Use tag instead of a parser function.Sep 7 2022, 8:35 AM

• JMcLeod_WMF added a project: Parsoid-Read-Views.Feb 23 2023, 4:36 PM

• JMcLeod_WMF moved this task from Uncategorized to Phase 1 - DiscussionTools support on the Parsoid-Read-Views board.Mar 2 2023, 3:19 PM

• JMcLeod_WMF edited projects, added Parsoid-Read-Views (Phase 1 - DiscussionTools support); removed Parsoid-Read-Views.

Looks like for wikimedia deployments, there are only 2 extensions that are impacted. CategoryTree (which Arlo has fixed up above) and WikimediaIncubator which needs fixing.

But, outside WMF, this feature is used heavily! So, the question is if we should try to fix this now OR after WMF switches over and we build Parsoid support for parser functions.

I did look into what is involved in fixing this. This will require a patch to Parser.php's insertStripItem to distinguish HTML strip items from others which is easy -- we can recover the isHTML flag from there.. It will also require us to change the DataAccess interface to pass through the 'isHTML' property all the way to TemplateHandler where we would need to wrap the output in a DocumentFragment, etc. just like ExtensionHandler.

Not too hard and doable.

But it may bew worth postponing this to post read views once we fix up WikimediaIncubator Extension as well. That extension is only used on incubatorwiki.

The infopage parser function which depends on this doesn't seem to be used as far as I can tell. So, for now, we can competely deprioritize this work till read views work on the wikimedia cluster is complete.

ssastry edited projects, added Parsoid-Read-Views; removed Parsoid-Read-Views (Phase 1 - DiscussionTools support).Mar 29 2023, 11:32 PM

Change 904628 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/core@master] [DNM]: Use special strip markers for HTML from parser functions

https://gerrit.wikimedia.org/r/904628

Change 904628 abandoned by Subramanya Sastry:

[mediawiki/core@master] [DNM]: Use special strip markers for HTML from parser functions

Reason:

https://gerrit.wikimedia.org/r/904628

Maintenance_bot removed a project: Patch-For-Review.Mar 30 2023, 9:30 PM

MSantos moved this task from Uncategorized to Phase 5: (Later) on the Parsoid-Read-Views board.Apr 13 2023, 2:22 PM

ssastry merged a task: T289545: Parsoid doesn't respect strip state markers found in preprocessor output.Apr 13 2023, 5:25 PM

ssastry merged a task: T178588: Handle parser functions returning raw html.