Page MenuHomePhabricator

Support MCR in mobile
Open, Needs TriagePublic

Description

The service stacks that support mobile (web/Android/iOS) need to support multi-content revisions (see especially the use cases there for background). This is an exploratory task to figure out the shape of that.

In the current PHP stack, some storage service looks up the content and metadata for the current revision, the content handler generates a HTML blob from it (in the case of wikitext this involves the parser and extension callbacks), the skin takes the content HTML and the revision metadata and renders the final page. For mobile, that is replaced by client-side rendering: the app fetches the revision metadata (via the PHP API or RESTBase/MCS), it fetches the content blob (via Parsoid/RESTBase/MCS, or by calling the PHP parser via the MobileFrontend API), and renders the final page.

In the new post-MCR PHP stack there will be more steps: the storage service looks up the revision metadata, and the slot metadata and content for each revision slot, the content handlers generate a HTML blob from each slot content, these slots get combined into a single HTML blob, and that gets handed along with revision metadata to the skin. So there will be a step that's largely analogous to the old rendering process (slot content -> slot HTML) and a new "combining" step (multiple pieces of slot HTML -> single multi-content HTML). Alternative renderings that don't use MediaWiki skin (and possibly parsing) logic will need to reproduce this.

(Note that there is significant uncertainty at this point around the "combining" step - it might pretty much just concatenation, or it might be more complex. It might be some transformation starting with a set of fully rendered slot HTMLs, or it might be more integrated with the parsing process.)

Specifically, this probably means that

  • Parsoid needs to be able to address single slots (i.e. handle routes like /page/html/<title>/<slot> and /page/html/<title>/<revision>/<slot>) and handle invalidation correctly (slots can depend on other slots of the same revision)
  • something (probably MCS?) in the Node stack needs to be able to combine a bunch of slot HTMLs into a single HTML so that apps can show a single-page rendering of any content they don't have any special handling for. This is probably going to be fairly simple (just wrap them in some HTML container and concatenate).
  • in RESTBase, /page/html/<title> and /page/html/<title>/<revision> should maybe redirect to that combining service. (Or maybe it should only show the main slot - not clear which would be the more sensible behavior for old clients.)
  • apps that use core MediaWiki API calls will have to implement HTML combining on their side, or the APIs might take care of that themselves when no slot parameter is present. I don't think there is any major app still using such calls.
  • for apps that use MobileFrontend's mobileview API, that API needs to do the HTML combining (there will be a PHP service for that).

Related Objects

StatusAssignedTask
Declineddchen
OpenNone
OpenNone
DuplicateNone
OpenNone
ResolvedAbit
OpenNone
OpenNone
OpenNone
OpenNone
DuplicateNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
Resolvedppelberg
ResolvedKrinkle
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone

Event Timeline

Tgr created this task.Apr 9 2018, 1:43 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Tgr updated the task description. (Show Details)Apr 9 2018, 1:45 PM
Tgr updated the task description. (Show Details)
Tgr updated the task description. (Show Details)Apr 9 2018, 1:49 PM
Tgr added a comment.Apr 9 2018, 1:52 PM

The other option that was discussed for how HTML generation works internally in MCR is that there will be a single HTML blob, which is initially the HTML rendering of the content of the main slot, and then handlers for the other slots somehow expand that progressively. That seems very hard to reproduce in the Node stack, though.

ssastry added a subscriber: ssastry.Apr 9 2018, 2:04 PM

Parsoid needs to be able to address single slots (i.e. handle routes like /page/html/<title>/<slot> and /page/html/<title>/<revision>/<slot>)

Bikeshedding:
(1) s/page/pageslot
(2) Better routes may be /pageslot/<slotid>/html/<title>(/<revision>)? or /pageslot/html/<slot>/<title>(/<revision>)? .. otherwise, there is ambiguity in the revision and slot parsing in RESTBase.

Tgr added a comment.Apr 9 2018, 2:07 PM

Slot IDs will be short strings (like main or documentation), not numbers. Although currently we don't verify that it's not a numeric string; we probably should.

Tgr updated the task description. (Show Details)Apr 9 2018, 2:09 PM
Anomie added a comment.Apr 9 2018, 2:09 PM
In T191795, @Tgr wrote:

Alternative renderings that don't use MediaWiki skin (and possibly parsing) logic will need to reproduce this.

At least for the first versions of MCR, skins aren't planned on being changed. But anything that gets "the" wikitext of a page to parse it itself (e.g. via Parsoid) will need to be able to deal with combining multiple slots in MCR.

The other option that was discussed for how HTML generation works internally in MCR is that there will be a single HTML blob, which is initially the HTML rendering of the content of the main slot, and then handlers for the other slots somehow expand that progressively. That seems very hard to reproduce in the Node stack, though.

That's not likely to be hard to reproduce in node, just annoying because of code duplication.

In T191795, @Tgr wrote:

the content handlers generate a HTML blob from each slot content, these slots get combined into a single HTML blob
[...]
and a new "combining" step (multiple pieces of slot HTML -> single multi-content HTML)

The details are what matter here.

  • Blindly concatenate the HTML blobs? Probably easy to duplicate in node.
  • Have some sort of per-page-type "template" into which the blobs get plugged in? You'd need to be able to get the right template into node. Which might be problematic if the "template" depends on the page metadata in some way.
  • Have the option to add different HTML than whatever blob the Content produces standalone?
  • Find markers of some sort in one slot's HTML to indicate where another's HTML is inserted?
  • Combinations of the above depending on the specific slots involved?
  • Something else?
In T191795, @Tgr wrote:

apps that use core MediaWiki API calls will have to implement HTML combining on their side (the APIs will support per-slot functionality). I don't think there is any major app still doing that.

Chances are that apps that use the action API action=parse and specify page, pageid, or oldid to get page content will just work, since the most likely implementation will be to return the combined HTML unless per-slot HTML is specifically requested (with a new parameter).

Anomie added a comment.Apr 9 2018, 2:10 PM

Slot IDs will be short strings (like main or documentation), not numbers. Although currently we don't verify that it's not a numeric string; we probably should.

Why should we? There's no reason a slot role name couldn't be "12345" if that makes sense in context.

Tgr added a comment.Apr 9 2018, 2:15 PM

Why should we? There's no reason a slot role name couldn't be "12345" if that makes sense in context.

I doubt there's a context in which that makes sense, and it seems like a useful guarantee for REST APIs. (Although such APIs might want to nevertheless avoid routes which only differ in what pattern the parameter matches, I suppose.)

ssastry added a comment.EditedApr 9 2018, 2:20 PM

something (probably MCS?) in the Node stack needs to be able to combine a bunch of slot HTMLs into a single HTML so that apps can show a single-page rendering of any content they don't have any special handling for. This is probably going to be fairly simple (just wrap them in some HTML container and concatenate).

In the general case, document composition will have to worry about https://www.mediawiki.org/wiki/User:SSastry_(WMF)/Notes/Document_Composability#Nesting_constraints ... But, in the simple case where the composition introduces the fragments as children of <body>, dealing with this will be the no-op constraint. Alternatively, we come up with a composition spec (see https://www.mediawiki.org/wiki/Parsing/Notes/HTML5#Composition_Spec_notes) that is applied uniformly in all html5 document composition scenarios everywhere in mediawiki.

Tgr updated the task description. (Show Details)Apr 9 2018, 2:20 PM
Tgr updated the task description. (Show Details)Apr 9 2018, 2:23 PM
bearND added a subscriber: bearND.Apr 9 2018, 3:25 PM
Niedzielski moved this task from Incoming to Tracking on the Readers-Web-Backlog board.
Niedzielski added a subscriber: Jdlrobson.
Niedzielski added a subscriber: pmiazga.
Niedzielski added a subscriber: Niedzielski.
daniel moved this task from Inbox to Epic on the Multi-Content-Revisions board.May 7 2018, 10:42 AM
daniel moved this task from Epic to Watching on the Multi-Content-Revisions board.May 7 2018, 10:45 AM
ssastry moved this task from Backlog to Non-Parsoid Tasks on the Parsoid board.Sep 20 2018, 5:15 PM

I just want to mention, that currently, the MobileFrontend allows fetching data from MCS instead of MediaWiki. Please refer to McsContentProvider class.
We're using it to verify how the HTML returned by /page/mobile-sections/{TITLE} looks on mobile. To test it locally please change the wgMFContentProviderClass to MobileFrontend\ContentProviders\McsContentProvider.
Also, the McsContentProvider allows to point it to different service, just change the wgMFMcsContentProviderBaseUri config variable (it's "https://en.wikipedia.org/api/rest_v1" by default)

WDoranWMF moved this task from MCR to mop on the Core Platform Team board.Fri, Jul 26, 6:38 PM