Page MenuHomePhabricator

Prepare MediaWiki for API-driven frontend
Open, NormalPublic

Description

This is a rough sketch to seek input from other developers.

Rationale

Enable MediaWiki to respond to page views in a fairly performant manner for all users (logged-in or not).

Outcomes
  • A site admin can enable static caching (eg. wgUseFileCache) and the cache will be used for both logged-out and logged-in users by having the minimal run-time still apply the skin after FileCache. (It is currently applied before FileCache, and anon-only.)
  • Routing logic will know which urls are page views and which not. (Currently special pages can disable OutputPage mid-way.)
  • The Skin is given (cacheable) data about the skin, the page and the user. It will produce its HTML output without performing additional queries. Therefore, any additional DB queries skins currently make for page-related data must instead happen in a way that is triggered earlier when the page is being cached (e.g. skin hooks for Parser, ParserCache or FileCache?)
Small but impactful changes
  1. MediaWiki will have a router that reliably know how to construct an output handler (e.g. no unpredictable OutputPage::disable() half-way through the request, such as in SpecialExport, RawAction and elsewhere).
  2. FileCache no longer caches entire HTML response. Instead it just stores the core page content and relevant data properties needed by the skin (e.g. display title, last modified date, categories, permissions, etc.).
  3. Skin Template must be programming-language neutral (e.g. Mustache template).
  4. Data flow in one direction only. No more two-way communication between Skin and OutputPage. OutputPage injects all relevant data (maybe some of it lazy-computed) into the Skin class. Skin class may combine it with User info and transform properties (renaming, deriving, etc.), and then invokes the Skin template with it.
  5. Skin hooks must change from being "callables with global context returning uncacheable raw html" to instead be "callables that return cacheable HTML or template partials". They can still vary by user and page (e.g. "What links here", "My contributions"), but the decision must be made in Mustache syntax using only the available user/page information. This way it can be rendered in a different server process, or even client-side, and still work and have all the information (T106099). It also means that on page views, all available meta data is naturally already fetched and cacheable. If extension hooks require additional information, they can do so, but must use hook behind the cache, not in front of it. E.g. to do their query when the page cache blob is generated. Not on-demand for every page view.

Other thoughts:

  • As things unfold, it might make sense to obsolete FileCache in favour of a better ParserOutput cache (if it has all the needed data and gains support for storing on-disk instead of in SQL). Given that the run-time overhead of taking a ParserOutput object and applying the Skin template would be cheap enough that it would be negligible. Alternatively, we may change FileCache to be more like a companion to ParserCache that caches only the extra data needed by skins. – Or store them in regular ObjectCache/mainCache instead.

Steps:

  • Deprecate OutputPage::disable().
  • Provide a router. This will map paths and query parameters to a handler, similar to MediaWiki::parseTitle() and other methods in MediaWiki.php do now. The decision to not use a skin or OutputPage for a response must be made here. The router registry should be in a static programming-language neutral format that can be exposed over the API for potential use by a different server process.
  • Make data flow one-direction between OutputPage and Skin.
  • Convert Skin hooks.
  • Convert one Skin (e.g. Vector) to become a template, as example. (Note, there is no pressure to remove support for PHP-based skins, the only requirement is that it can handle responses in front of the page cache instead of behind it, e.g. using data properties instead of run-time database queries and WikiPage methods calls).

What this task is not

This task does not change the architecture of MediaWiki or the skin system. It also does not change how MediaWiki is used or deployed at WMF.

This work covered by this task resolves long-standing technical debt in MediaWiki for the purposes of improved rendering performance and reduced overall complexity and maintenance cost for the skin system – whilst remaining fully backwards compatible.

The intended result is improved performance of rendering page views in MediaWiki core, both out-of-the-box and at WMF.

The works covered by this task should in my opinion be considered a prerequisite for the following proposals. Some of the below might be possible without this task, although I believe doing so would incur significant amounts of maintenance overhead and technical debt that can and should be avoided (through this task).

  • Add a way for MediaWiki to render wiki page, actions, and special pages without a skin. – T114596
  • Re-implement MediaWiki skin rendering for WMF-deployed skins in a micro-service that runs near the CDN. – T111588
  • Re-implement MediaWiki skin rendering for WMF-deployed skins in a way that can be run offline and client-side using service workes. – T106099.

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
brion added a subscriber: brion.Aug 17 2016, 7:01 PM

Change 372471 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] Skin: Remove redundant use of QuickTemplate::setRef()

https://gerrit.wikimedia.org/r/372471

Change 372471 merged by jenkins-bot:
[mediawiki/core@master] Skin: Remove redundant use of QuickTemplate::setRef()

https://gerrit.wikimedia.org/r/372471

I feel like it might be relevant..

A while back I experimented with reducing skins to a template and json file:
https://github.com/jdlrobson/SimpleSkins/tree/master/skins

My goal was to prevent skin designers from having to know PHP and to concentrate on a frontend stack they were familiar with.
It worked pretty well and I managed to rebuild Vector, Minerva and Monobook this way while making these skins as dumb as possible (they just rendered what they were given)

I think it also made API driven frontends more of a possibility but I never worked out how to fit in with the extension system e.g. hooks - my feeling was that a feature like Echo would have to be integrated by the skin developer - not the Echo extension developer, or a placeholder interface would need to be left in core that Echo would need to implement.

Change 373356 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] Skin: Remove more use of QuickTemplate::setRef()

https://gerrit.wikimedia.org/r/373356

Change 373356 merged by jenkins-bot:
[mediawiki/core@master] Skin: Remove more use of QuickTemplate::setRef()

https://gerrit.wikimedia.org/r/373356

Change 386327 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] API: Include setupSkinUserCss in prop=modules for useskin mode

https://gerrit.wikimedia.org/r/386327

Change 386327 merged by jenkins-bot:
[mediawiki/core@master] API: Include setupSkinUserCss in prop=modules for useskin mode

https://gerrit.wikimedia.org/r/386327

Change 406312 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/skins/MinervaNeue@master] Remove use of QuickTemplate::setRef

https://gerrit.wikimedia.org/r/406312

Change 406313 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/extensions/Polyglot@master] Remove use of QuickTemplate::setRef

https://gerrit.wikimedia.org/r/406313

Change 406315 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/extensions/StickToThatLanguage@master] Remove use of QuickTemplate::setRef

https://gerrit.wikimedia.org/r/406315

Change 406316 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] skins: Deprecate QuickTemplate::setRef()

https://gerrit.wikimedia.org/r/406316

Krinkle updated the task description. (Show Details)Jan 26 2018, 6:23 PM

Change 406318 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/skins/Vector@master] Remove 'SkinVectorStyleModules' hook

https://gerrit.wikimedia.org/r/406318

Change 406313 merged by jenkins-bot:
[mediawiki/extensions/Polyglot@master] Remove use of QuickTemplate::setRef

https://gerrit.wikimedia.org/r/406313

Change 406315 merged by jenkins-bot:
[mediawiki/extensions/StickToThatLanguage@master] Remove use of QuickTemplate::setRef

https://gerrit.wikimedia.org/r/406315

Change 406312 merged by jenkins-bot:
[mediawiki/skins/MinervaNeue@master] Remove use of QuickTemplate::setRef

https://gerrit.wikimedia.org/r/406312

Change 406318 merged by jenkins-bot:
[mediawiki/skins/Vector@master] Remove 'SkinVectorStyleModules' hook

https://gerrit.wikimedia.org/r/406318

Imarlier moved this task from Inbox to Radar on the Performance-Team board.Jan 30 2018, 9:02 PM
Imarlier edited projects, added Performance-Team (Radar); removed Performance-Team.

Change 406316 merged by jenkins-bot:
[mediawiki/core@master] skins: Deprecate QuickTemplate::setRef()

https://gerrit.wikimedia.org/r/406316

Krinkle triaged this task as Normal priority.Feb 2 2018, 10:55 PM
Krinkle moved this task from Limbo to Watching on the Performance-Team (Radar) board.
Krinkle edited projects, added Performance-Team; removed Performance-Team (Radar).
Krinkle moved this task from Inbox to Backlog: Small & Maintenance on the Performance-Team board.

Change 428856 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] skins: Move default style modules to getDefaultModules

https://gerrit.wikimedia.org/r/428856

Change 430830 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] API: Add ApiParseTest case for 'styles' in getDefaultModules

https://gerrit.wikimedia.org/r/430830

Krinkle updated the task description. (Show Details)May 4 2018, 3:46 AM
Krinkle updated the task description. (Show Details)

Change 430830 merged by jenkins-bot:
[mediawiki/core@master] API: Add ApiParseTest case for 'styles' in getDefaultModules

https://gerrit.wikimedia.org/r/430830

Change 428856 merged by jenkins-bot:
[mediawiki/core@master] skins: Move default style modules to getDefaultModules

https://gerrit.wikimedia.org/r/428856

Nirmos added a subscriber: Nirmos.May 18 2018, 1:41 AM
Izno updated the task description. (Show Details)Sep 8 2018, 9:31 PM

Change 460406 had a related patch set uploaded (by Krinkle; owner: C. Scott Ananian):
[mediawiki/core@master] WIP: hard-deprecate adding modules from BeforePageDisplay hook

https://gerrit.wikimedia.org/r/460406

Change 485998 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/skins/Vector@master] Start extracting rendering from PHP into Mustache

https://gerrit.wikimedia.org/r/485998

Change 485998 merged by jenkins-bot:
[mediawiki/skins/Vector@master] Start extracting rendering from PHP into Mustache

https://gerrit.wikimedia.org/r/485998

Isarra added a subscriber: Isarra.Jan 24 2019, 9:48 PM
Krinkle updated the task description. (Show Details)Jan 26 2019, 6:50 AM

Change 485998 merged by jenkins-bot:
[mediawiki/skins/Vector@master] Start extracting rendering from PHP into Mustache https://gerrit.wikimedia.org/r/485998

(Patch was reverted in https://gerrit.wikimedia.org/r/c/mediawiki/skins/Vector/+/486874)

Izno added a subscriber: Izno.Mar 12 2019, 11:36 PM
Krinkle updated the task description. (Show Details)Sep 18 2019, 3:59 PM