Page MenuHomePhabricator

Improve performance when writing multi-content revisions to XML dumps
Closed, ResolvedPublic

Description

TextPassDumper uses a pre-fetch mechanism to improve performance when writing revision text. This mechanism needs to be adapted for use with multi-content revisions.

Event Timeline

daniel created this task.Feb 25 2020, 9:12 AM

Change 556346 had a related patch set uploaded (by Daniel Kinzler; owner: Daniel Kinzler):
[mediawiki/core@master] TextPassDumper: implement prefetch for non-main slots.

https://gerrit.wikimedia.org/r/556346

Change 556344 had a related patch set uploaded (by Daniel Kinzler; owner: Daniel Kinzler):
[mediawiki/core@master] XmlDumpWriter: fix blob address output in stubs

https://gerrit.wikimedia.org/r/556344

daniel triaged this task as High priority.Feb 25 2020, 9:17 AM
daniel added a project: MW-1.35-release.

Bumping to high, since MCR support for dumps should be in 1.35.

Note that the patches for this task do not have full test coverage. They are however covered by the patch proposed for T240213: Write integration tests for XML dumps with multiple MCR slots per revision.

ArielGlenn moved this task from Backlog to Active on the Dumps-Generation board.Feb 25 2020, 9:28 AM

Change 556344 merged by jenkins-bot:
[mediawiki/core@master] XmlDumpWriter: fix blob address output in stubs

https://gerrit.wikimedia.org/r/556344

Change 556346 merged by jenkins-bot:
[mediawiki/core@master] TextPassDumper: implement prefetch for non-main slots.

https://gerrit.wikimedia.org/r/556346

CCicalese_WMF closed this task as Resolved.Apr 28 2020, 8:48 PM
CCicalese_WMF added a subscriber: CCicalese_WMF.

Marking as Resolved as it is in the Done column. Feel free to reopen if there is remaining work.

ArielGlenn moved this task from Other teams to Done on the Dumps-Generation board.May 20 2020, 8:09 AM