Page MenuHomePhabricator
Create Task
Maniphest T90374

Retrieve page HTML directly from RESTbase
Closed, ResolvedPublic8 Estimated Story Points
Actions

Description

Retrieving the page HTML via api.php is probably inefficient. We've observed that the time to first byte is very high (over a second, sometimes two) and the time spent transmitting data very low (a few hundred milliseconds). Additionally, the API wraps the page HTML in a JSON blob which we then have to decode.

Instead, we should send two separate requests, one for the page HTML directly to RESTbase, and one for a JSON blob of auxiliary information to api.php.

Details

SubjectRepoBranchLines +/-
Make VisualEditor access RESTbase directly on Wikipediasoperations/mediawiki-configmaster+1 -0
Make VisualEditor access RESTbase directly on enwikioperations/mediawiki-configmaster+1 -0
Make VisualEditor access RESTbase directly in group0operations/mediawiki-configmaster+19 -0
Send requests to RESTbase directly when so configuredmediawiki/extensions/VisualEditormaster+123 -79
Customize query in gerrit

Related Objects
Search...

View Standalone Graph
This task is connected to more than 200 other tasks. Only direct parents and subtasks are shown here. Use View Standalone Graph to show more of the graph.
StatusSubtypeAssignedTask
· · ·
ResolvedCatropeT90374 Retrieve page HTML directly from RESTbase
Resolved• GWickeT89481 RESTBase beta release (revision storage / content API)
Resolved• mobrovacT91102 Update / maintain Beta Cluster restbase cluster: Up & working with VE
ResolvedKrenairT92993 Handle page content data being for a different revision compared to the API-added data
Resolved• GWickeT93452 Enable group 1 wikis in RESTBase
· · ·

Event Timeline

Catrope created this task.Feb 22 2015, 6:36 AM
Catrope claimed this task.
Catrope raised the priority of this task from to Needs Triage.
Catrope updated the task description. (Show Details)
Catrope added projects: VisualEditor-Performance, VisualEditor, VisualEditor 2014/15 Q3 blockers.
Catrope added subscribers: • GWicke, ori.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 22 2015, 6:36 AM
Catrope added a subtask: T89481: RESTBase beta release (revision storage / content API).Feb 22 2015, 6:38 AM
Catrope added a comment.Feb 22 2015, 6:41 AM

Blocked on RESTbase being fully deployed and reliable. We can't use parsoid-lb.eqiad.wikimedia.org for this in its current form because it doesn't serve CORS headers and appears to have no or nonfunctional caching (responses have X-Varnish headers, but they're always misses).

Catrope set Security to None.Feb 22 2015, 6:41 AM
Catrope edited a custom field.
Liuxinyu970226 subscribed.Feb 22 2015, 6:42 AM
Krenair subscribed.Feb 22 2015, 4:52 PM

Do we have a restbase instance in labs to write code against?

Jdforrester-WMF triaged this task as High priority.Feb 23 2015, 6:16 PM
Jdforrester-WMF moved this task from To Triage to Bug Fixes on the VisualEditor board.
• GWicke added a comment.Feb 23 2015, 6:56 PM

@Krenair, see https://rest.wikimedia.org/en.wikipedia.org/v1/?doc. This went alpha-live on Friday & needs a little bit more testing, but will be officially ready this week.

See also:

Jdforrester-WMF moved this task from Nominated to Accepted on the VisualEditor 2014/15 Q3 blockers board.Feb 26 2015, 12:27 AM
Jdforrester-WMF added a parent task: T90304: Load HTML in parallel with JSON metadata.Mar 10 2015, 10:47 PM
Catrope mentioned this in T88623: On Parsoid HTML read pages, VisualEditor should be able to load data-mw from a separate API call and zip it into the DOM.Mar 11 2015, 3:12 PM
• GWicke closed subtask T89481: RESTBase beta release (revision storage / content API) as Resolved.Mar 13 2015, 1:33 AM
gerritbot subscribed.Mar 13 2015, 11:42 PM

Change 196720 had a related patch set uploaded (by Catrope):
[WIP] Send requests to RESTbase directly when so configured

https://gerrit.wikimedia.org/r/196720

gerritbot added a project: Patch-For-Review.Mar 13 2015, 11:42 PM
Jdforrester-WMF moved this task from Bug Fixes to Blocked on the VisualEditor board.Mar 16 2015, 6:38 PM
Jdforrester-WMF added a parent task: T87553: Performance issue in HHVM with output buffer?.Mar 16 2015, 8:07 PM
Jdforrester-WMF removed a parent task: T90304: Load HTML in parallel with JSON metadata.
Catrope merged a task: T87553: Performance issue in HHVM with output buffer?.Mar 17 2015, 6:54 PM
Catrope removed a parent task: T87553: Performance issue in HHVM with output buffer?.
Catrope added subscribers: • mobrovac, Jdforrester-WMF, • Elitre and 2 others.
Catrope added a subtask: T92993: Handle page content data being for a different revision compared to the API-added data.Mar 17 2015, 7:24 PM
Krenair mentioned this in T92993: Handle page content data being for a different revision compared to the API-added data.Mar 19 2015, 3:16 AM
Krenair added a comment.Mar 19 2015, 3:29 AM

Before we merge this we should have RESTbase in beta to test against, and preferably group1 wikis (which we've managed to skip deploying RESTbase config to, somehow). Probably needs separate tasks.

• mobrovac added a subtask: T91102: Update / maintain Beta Cluster restbase cluster: Up & working with VE.Mar 19 2015, 2:00 PM
In T90374#1131020, @Krenair wrote:

Before we merge this we should have RESTbase in beta to test against, and preferably group1 wikis (which we've managed to skip deploying RESTbase config to, somehow). Probably needs separate tasks.

T91102 is in the works, should be completed pretty soon.

• GWicke added a comment.Mar 20 2015, 11:45 PM
In T90374#1131020, @Krenair wrote:

Before we merge this we should have RESTbase in beta to test against, and preferably group1 wikis (which we've managed to skip deploying RESTbase config to, somehow). Probably needs separate tasks.

Group 1 deploy is tracked in T93452. There is a puppet patch for it already.

Jdforrester-WMF added a subtask: T93452: Enable group 1 wikis in RESTBase.Mar 20 2015, 11:46 PM
• mobrovac closed subtask T91102: Update / maintain Beta Cluster restbase cluster: Up & working with VE as Resolved.Mar 24 2015, 2:26 PM
Jdforrester-WMF closed subtask T92993: Handle page content data being for a different revision compared to the API-added data as Resolved.Mar 25 2015, 8:51 PM
Catrope edited a custom field.Mar 26 2015, 9:17 PM
gerritbot added a comment.Mar 26 2015, 11:16 PM

Change 196720 merged by jenkins-bot:
Send requests to RESTbase directly when so configured

https://gerrit.wikimedia.org/r/196720

Catrope mentioned this in rEVEDb19104ed04dd: Send requests to RESTbase directly when so configured.Mar 26 2015, 11:17 PM
gerritbot added a comment.Mar 27 2015, 2:36 AM

Change 200098 had a related patch set uploaded (by Jforrester):
[WIP] Make VisualEditor access RESTbase directly

https://gerrit.wikimedia.org/r/200098

gerritbot added a comment.Mar 27 2015, 5:08 AM

Change 200105 had a related patch set uploaded (by Jforrester):
Make VisualEditor access RESTbase directly on enwiki

https://gerrit.wikimedia.org/r/200105

gerritbot added a comment.Mar 27 2015, 5:08 AM

Change 200106 had a related patch set uploaded (by Jforrester):
Make VisualEditor access RESTbase directly on Wikipedias

https://gerrit.wikimedia.org/r/200106

gerritbot added a comment.Mar 27 2015, 5:08 AM

Change 200107 had a related patch set uploaded (by Jforrester):
Make VisualEditor access RESTbase directly on all wikis

https://gerrit.wikimedia.org/r/200107

Jdforrester-WMF moved this task from Accepted to Done on the VisualEditor 2014/15 Q3 blockers board.Mar 27 2015, 8:10 PM
Jdforrester-WMF closed this task as Resolved.Mar 28 2015, 12:49 AM
Liuxinyu970226 unsubscribed.Mar 28 2015, 1:00 AM
Jdforrester-WMF moved this task from Blocked to FY 18-19 Q3/Q4 on the VisualEditor board.Mar 29 2015, 10:47 PM
gerritbot added a comment.Apr 1 2015, 11:41 PM

Change 200098 merged by jenkins-bot:
Make VisualEditor access RESTbase directly in group0

https://gerrit.wikimedia.org/r/200098

Catrope mentioned this in rOMWCe0f2643012a4: Make VisualEditor access RESTbase directly in group0.Apr 1 2015, 11:42 PM
gerritbot added a comment.Apr 8 2015, 9:07 PM

Change 200105 merged by jenkins-bot:
Make VisualEditor access RESTbase directly on enwiki

https://gerrit.wikimedia.org/r/200105

Catrope mentioned this in rOMWC9362d28e5ff7: Make VisualEditor access RESTbase directly on enwiki.Apr 8 2015, 9:08 PM
• mobrovac added a comment.Apr 8 2015, 9:15 PM
In T90374#1191779, @gerritbot wrote:

Change 200105 merged by jenkins-bot:
Make VisualEditor access RESTbase directly on enwiki

https://gerrit.wikimedia.org/r/200105

yipikaye!

gerritbot added a comment.Apr 8 2015, 11:18 PM

Change 200106 merged by jenkins-bot:
Make VisualEditor access RESTbase directly on Wikipedias

https://gerrit.wikimedia.org/r/200106

Catrope mentioned this in rOMWC107a92392c7f: Make VisualEditor access RESTbase directly on Wikipedias.Apr 8 2015, 11:19 PM
• GWicke added a comment.Apr 9 2015, 4:00 AM

Request latency on enwiki looked great, but was really bad on other wikis. We rolled back the all-wikipedia change for this reason. See https://phabricator.wikimedia.org/T95229#1192818 for the details.

• MZMcBride subscribed.Apr 9 2015, 4:01 AM
• GWicke closed subtask T93452: Enable group 1 wikis in RESTBase as Resolved.May 20 2015, 5:35 AM
Jdforrester-WMF removed a project: Patch-For-Review.Jun 10 2015, 6:09 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL