Page MenuHomePhabricator

LinkCache should be smarter
Closed, ResolvedPublic1 Estimated Story Points

Description

Right now ApiVisualEditor::execute has to output a "foo-title": {"missing": false} object for every non-red link on the page. If we simply omit non-red links, VeInitMwLinkCache will assume that the response from the API was incomplete, and will queue up API requests to resolve any links that weren't explicitly described by the initial paction=parse API response.

This is frightfully inefficient. A more sensible approach would be as follows:

  • On parser cache miss, omit the 'links' key entirely. The client-side LinkCache should take that to mean that all links will have to be looked up via the API.
  • On parser cache hit, the 'links' key should map to an array of titles that are red links. If there aren't any, the array should be present but empty. Non-red-links should not be in the array at all.

Event Timeline

ori created this task.Feb 2 2015, 5:10 AM
ori raised the priority of this task from to Needs Triage.
ori updated the task description. (Show Details)
ori added a project: VisualEditor-Performance.
ori added subscribers: ori, Catrope.
Restricted Application added a project: VisualEditor. · View Herald TranscriptFeb 2 2015, 5:10 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
ori added a comment.Feb 2 2015, 5:13 AM

The relevant code is in ApiVisualEditor.php, L462-484 and in ve.init.mw.LinkCache.js.

Counter-proposal: for current revisions, let's enumerate only the red links and assume all other links are blue. This places slightly more trust in Parsoid and ParserOutput agreeing with each other than I'm comfortable with, but it's just presentation at the end of the day. For old revisions, let's continue enumerating both the red and blue links of the current revision, betting on a significant overlap between link targets in the current revision and the old one, but recognizing that the old revision might contain links that aren't in the current one so we can't just assume that those links will be either red or blue based on the data we have to work with.

Catrope claimed this task.Feb 3 2015, 3:41 AM
gerritbot added a subscriber: gerritbot.

Change 188297 had a related patch set uploaded (by Catrope):
Make API module's LinkCache data transmission format more efficient

https://gerrit.wikimedia.org/r/188297

Patch-For-Review

Jdforrester-WMF triaged this task as High priority.Feb 3 2015, 5:55 PM
Jdforrester-WMF set Security to None.
Jdforrester-WMF edited a custom field.

Change 188297 merged by jenkins-bot:
Make API module's LinkCache data transmission format more efficient

https://gerrit.wikimedia.org/r/188297

Jdforrester-WMF moved this task from Blocked to Q4 on the VisualEditor board.Feb 4 2015, 4:03 PM
ori moved this task from Backlog to Done on the VisualEditor-Performance board.Feb 19 2015, 2:29 AM