Page MenuHomePhabricator

Parsoid performance: Use RESTBase from the MediaWiki Virtual Rest Service on group1/group2 wikis
Closed, ResolvedPublic

Description

The Virtual Rest Service for Parsoid content currently uses the Parsoid v1 API. Once RESTBase is deployed and well tested we should start switching this over to RESTBase, so that VisualEditor, Flow and other consumers can profit from the smaller HTML without inline data-parsoid & faster load times.

For a smooth migration we should support both backends & have the option to switch between them using a config flag in MediaWiki. Both infrastructures (RESTBase and Parsoid Varnishes) will be available in parallel during the transition period.

This is also a good opportunity to clean up some things in this area:

  • share / generalize the config & setup for VRS & Parsoid / RESTBase: factory function / singleton? (Could perhaps take a similar approach Ori took in his statsd patch.)
  • Adjust URL schema to current restbase layout (/page/html/{title}/{revision}) and mount it at /local/ for now. In the longer term, we'll want to support a domain at that level, but for now we only need to support /local/.
  • eventually, remove the /parsoid/ prefix (requires some changes in the VRS base class)

Related Objects

StatusAssignedTask
Resolved GWicke
Resolvedmobrovac
Resolvedmobrovac
ResolvedNone
Resolved GWicke
Resolved GWicke
ResolvedNone
Resolved GWicke
Resolvedmobrovac
Resolved GWicke
Resolved GWicke
Resolvedfgiunchedi
Resolvedfgiunchedi
ResolvedCmjohnson
ResolvedCmjohnson
ResolvedJoe
Resolvedfgiunchedi
Resolved GWicke
Resolved Jdouglas
Resolved GWicke
Resolved GWicke
ResolvedArlolra
Resolved GWicke
Resolvedmobrovac
Resolvedmobrovac
Resolvedmobrovac
Resolvedmobrovac
Duplicate Jdouglas
ResolvedAndrew
Resolved GWicke
Resolvedfgiunchedi
Resolvedfgiunchedi
Resolvedfgiunchedi
ResolvedEevans
Resolvedfgiunchedi
Resolved GWicke
Resolved GWicke
Resolvedfgiunchedi
Resolvedmobrovac
Resolved GWicke

Event Timeline

GWicke raised the priority of this task from to Needs Triage.
GWicke updated the task description. (Show Details)
GWicke added a project: MediaWiki-General.
GWicke added a subscriber: GWicke.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 10 2015, 1:21 AM
GWicke updated the task description. (Show Details)Feb 10 2015, 1:23 AM
GWicke set Security to None.
GWicke added a project: Services.
GWicke triaged this task as Normal priority.Feb 10 2015, 1:25 AM
GWicke added a subscriber: mobrovac.EditedFeb 26 2015, 4:57 PM

Some recent discussion on the old gerrit patch for parsoid integration: https://gerrit.wikimedia.org/r/#/c/181011/12/ApiVisualEditor.php

GWicke raised the priority of this task from Normal to High.Feb 26 2015, 5:03 PM
GWicke updated the task description. (Show Details)
GWicke updated the task description. (Show Details)Feb 26 2015, 5:10 PM
mobrovac claimed this task.Feb 27 2015, 2:11 PM
GWicke updated the task description. (Show Details)Feb 27 2015, 4:34 PM
greg added a subscriber: greg.Feb 27 2015, 6:22 PM

Where are we on the timeline for this? (Deployment calendar says "late feb early march")

@greg should be right on time. We are contemplating a possible multi-step solution where the first step would be to refactor things in core so that it's possible to switch from Parsoid to RESTBase without making changes in VE itself. This is somewhat a messier solution, but I'd allow us to start testing VE with RESTBase pretty soon.

Change 193826 had a related patch set uploaded (by Mobrovac):
Introduce the Restbase Virtual REST Service class

https://gerrit.wikimedia.org/r/193826

Change 194338 had a related patch set uploaded (by Mobrovac):
Use the RESTBase back-end if available

https://gerrit.wikimedia.org/r/194338

greg moved this task from Unscheduled to March 2015: Platform on the Roadmap board.
greg moved this task from March 2015: Platform to March 9-13 on the Roadmap board.

@mobrovac, I'll amend your patch to address some of the comments today.

I hope that we can get this in early next week, so that we can test with VE in the second half of the week.

Eloquence renamed this task from Use RESTBase from the MediaWiki Virtual Rest Service to VisualEditor performance: Use RESTBase from the MediaWiki Virtual Rest Service.Mar 6 2015, 7:37 PM

This is about all Parsoid HTML consumers of the MediaWiki core service, not just VisualEditor, isn't it? Flow, etc.

Eloquence renamed this task from VisualEditor performance: Use RESTBase from the MediaWiki Virtual Rest Service to Parsoid performance: Use RESTBase from the MediaWiki Virtual Rest Service.Mar 7 2015, 7:09 PM

The patches for MW core and for VE have been tested on a local MW-Vagrant install and work flawlessly: VE gets the HTML from RESTBase on load and passes it back for transformation.

Change 193826 merged by jenkins-bot:
Introduce the Restbase Virtual REST Service class

https://gerrit.wikimedia.org/r/193826

Change 194338 merged by jenkins-bot:
Use the RESTBase back-end if available

https://gerrit.wikimedia.org/r/194338

GWicke reopened this task as Open.EditedMar 12 2015, 12:21 AM

Re-opening as this is not fully done yet.

The integration is now being tested on https://test.wikipedia.org/. Things are working fine, although performance on test.wikipedia.org seems to be a bit mixed. VE initializes quickly using RESTBase-provided content, but saves seem to be relatively slow in both VE and the wikitext editor.

Next step (possibly tomorrow) is to enable this across phase0 wikis, eventually followed by real production.

VE is now using RESTBase on all group0 wikis including https://www.mediawiki.org. Further roll-out to follow next week.

GWicke renamed this task from Parsoid performance: Use RESTBase from the MediaWiki Virtual Rest Service to Parsoid performance: Use RESTBase from the MediaWiki Virtual Rest Service: phase0 done, others to follow.Mar 13 2015, 1:36 AM
Eloquence renamed this task from Parsoid performance: Use RESTBase from the MediaWiki Virtual Rest Service: phase0 done, others to follow to Parsoid performance: Use RESTBase from the MediaWiki Virtual Rest Service on phase1/phase2 wikis.Mar 13 2015, 4:47 PM
Eloquence moved this task from March 9-13 to March 16-20 on the Roadmap board.
Krenair renamed this task from Parsoid performance: Use RESTBase from the MediaWiki Virtual Rest Service on phase1/phase2 wikis to Parsoid performance: Use RESTBase from the MediaWiki Virtual Rest Service on group1/group2 wikis.Mar 13 2015, 5:56 PM
GWicke added a comment.EditedMar 16 2015, 6:13 PM

Further roll-out is starting now:

  1. ptwiki and ruwiki: done
  2. itwiki, plwiki: done
  3. frwiki: done
  4. remaining wikipedias: done

Edit: It turns out that none of these actually had an effect, as the VE extension in wmf20 did not include the RESTBase integration yet. It is going to be updated with the train on Wednesday.

Change 197104 had a related patch set uploaded (by GWicke):
Enable RESTBase for visual editing on itwiki and plwiki as well

https://gerrit.wikimedia.org/r/197104

Change 197104 merged by jenkins-bot:
Enable RESTBase for visual editing on itwiki and plwiki as well

https://gerrit.wikimedia.org/r/197104

Change 197108 had a related patch set uploaded (by GWicke):
Use RESTBase with VisualEditor on frwiki

https://gerrit.wikimedia.org/r/197108

Change 197108 merged by jenkins-bot:
Use RESTBase with VisualEditor on frwiki

https://gerrit.wikimedia.org/r/197108

GWicke added a comment.EditedMar 16 2015, 11:31 PM

It turns out that the VE extension deployed in wmf20 does not actually have the restbase integration code yet, so the config changes on *.wikipedia.org were basically no-ops.

Patch to disable the setting until the train deploy on Wednesday updates the VE extension to wmf21 at https://gerrit.wikimedia.org/r/#/c/197250/.

Change 197250 had a related patch set uploaded (by Anomie):
Disable restbase on wikipedias

https://gerrit.wikimedia.org/r/197250

Change 197250 merged by jenkins-bot:
Disable restbase on wikipedias

https://gerrit.wikimedia.org/r/197250

GWicke moved this task from Backlog to In-progress on the Services board.Mar 17 2015, 8:03 PM
GWicke moved this task from Backlog to In progress on the RESTBase board.Mar 17 2015, 8:21 PM
GWicke added a comment.EditedMar 19 2015, 5:56 PM

As of now RESTBase is powering VisualEditor on all wikipedias. First performance metrics are looking promising:

GWicke closed this task as Resolved.Mar 19 2015, 6:02 PM

Lets track group1 in a follow-up task. Closing this task as resolved.

mobrovac moved this task from In-progress to Done on the Services board.Jun 23 2015, 8:49 AM
Restricted Application added a project: Notice. · View Herald TranscriptJun 23 2015, 8:49 AM
gpaumier moved this task from Backlog to Triaged on the Notice board.Jun 25 2015, 3:13 PM
gpaumier moved this task from Triaged to Archive on the Notice board.Jun 26 2015, 8:27 PM