Page MenuHomePhabricator

Expand templates in Parsoid instead of using MediaWiki core preprocessor
Open, LowPublic


Parsoid has support for expanding templates natively which we currently use in running parser tests in Parsoid. However, in production, we use the core preprocessor. To enable this in production, a bunch of issues have to be effectively addressed:

  1. Implement SOL newline hack T241150: Parsoid native template expansion can't handle the SOL newline hack
  2. Ensure the parser functions extension can run in Parsoid with Parsoid's token stream. We do have a very incomplete version of this extension in Parsoid (from the Parsoid/JS days) but it might be better to look at how to leverage the code parser functions extension code as much as possible.
  3. Ensure we can hook into Scribunto properly
  4. Complete support for setFunctionHook in Parsoid (T268144: Add setFunctionHook equivalent support to Parsoid Extension API)
  5. Implement proper caching support in template expansion.
  6. But, the biggest issue is likely going to be breakage and incompatibilities from token/string gluing that might be going on in native wikitext (non-Lua-module) templates. We will need to get a handle on the scope of this problem and see what is involved.

Given that problems 3 and 4 will be solved relatively soon and given that 1, 2, 5 are probably relatively not that hard, we are basically left with the unknown scope of item 6 above. ven so, this is still a task that will consume couple quarters of someone's focused time.

For now, using core's preprocessor is working out fine for the most part even if there are some mild performance inefficiencies (having transclusions get tokenized in Parsoid as well as by the preprocessor). Our current strategy isolates wikitext-based templates as their own special thing, and if Lua modules and wikifunctions supplant the use of the native wikitext templates, we may never have to deal with this.

This is mostly a low-priority placeholder if / when we get around to doing this.

Event Timeline

ssastry created this task.
ssastry renamed this task from Expands templates in Parsoid instead of using MediaWiki core preprocessor to Expand templates in Parsoid instead of using MediaWiki core preprocessor.Feb 11 2022, 4:17 PM