Allow Wikibase Entities to be stored in alternative slots [MCR]
Open, Needs TriagePublic

Description

With Multi-Content-Revisions, MediaWiki allows different kinds of content to be stored on the same page. To enable MediaWiki-extensions-WikibaseMediaInfo to store MediaInfo entities on file description pages in an extra slot, alongside the wikitext description, Wikibase needs to be changed to use the new page update (T174038) and revision retrieval (T174024) interfaces.

In concrete terms, this means:

  1. Change WikiPageEntityStore to use the new storage facilities in core (RevisionStore and PageUpdater) instead of Revision.
    1. WikiPageEntityRevisionLookup::loadEntity should now use RevisionStore. loadEntityBlob() should no longer be needed, since RevisionStore supports cross-wiki revision loading. However, this has not bee tested in production, and may require a RevisionStoreFactory to be introduced.
    2. WikiPageEntityMetaDataLookup also needs to be changed to make use of the new DB schema (ideally avoiding any low level access or knowledge of the schema). In particular, selectRevisionInformationById() can no longer directly access the text table.
  2. Define and pass through configuration that specifies in which slot of which namespace a given entity type resides (instead of just specifying the namespace).
There are a very large number of changes, so older changes are hidden. Show Older Changes
Restricted Application added a project: Wikidata. · View Herald TranscriptMay 15 2018, 8:38 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
daniel assigned this task to Addshore.Jun 26 2018, 3:22 PM
Restricted Application added a project: User-Addshore. · View Herald TranscriptJun 26 2018, 3:22 PM

Change 442253 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/Wikibase@master] WIP DNM Use PageUpdater & RevisionStore in WikiPageEntityStore

https://gerrit.wikimedia.org/r/442253

Addshore moved this task from Backlog to In Progress on the User-Addshore board.Jun 27 2018, 11:33 AM
daniel added a comment.EditedJun 27 2018, 11:53 AM

Oh, this is missing the requirements for loading entities! I have updated the task description. I'm afraid the reading part needs more work.

daniel updated the task description. (Show Details)Jun 27 2018, 11:57 AM

Change 442285 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/core@master] WIP Introduce RevisionStoreFactory

https://gerrit.wikimedia.org/r/442285

Ladsgroup moved this task from incoming to monitoring on the Wikidata board.Jun 28 2018, 3:38 PM

Change 442285 merged by jenkins-bot:
[mediawiki/core@master] Introduce RevisionStoreFactory & Tests

https://gerrit.wikimedia.org/r/442285

Vvjjkkii renamed this task from Allow Wikibase Entities to be stored in alternative slots [MCR] to 6xcaaaaaaa.Jul 1 2018, 1:09 AM
Vvjjkkii triaged this task as High priority.
Vvjjkkii removed Addshore as the assignee of this task.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii edited subscribers, added: Addshore; removed: gerritbot, Aklapper.
daniel renamed this task from 6xcaaaaaaa to Allow Wikibase Entities to be stored in alternative slots [MCR].Jul 2 2018, 2:46 PM
daniel lowered the priority of this task from High to Normal.
daniel updated the task description. (Show Details)

I started work on this but now have no more time as I'll be working on Wikidata-Senses-Hike & Wikidata-Senses-Iteration1

I'll file some subtickets / tasks now regarding the state

The patch in Wikibase of https://gerrit.wikimedia.org/r/442253 is basically ready once T198701 is complete

Addshore moved this task from In Progress to Watching on the User-Addshore board.Jul 3 2018, 12:04 PM
CommunityTechBot assigned this task to Addshore.
CommunityTechBot raised the priority of this task from Normal to Needs Triage.
CommunityTechBot renamed this task from Allow Wikibase Entities to be stored in alternative slots [MCR] to Allow Wikibase Entities to be stored in alternative slots [MCR].
daniel added a comment.Aug 1 2018, 4:16 PM

The patches that are up for review at this point do not fully cover the necessary functionality. Most importantly, there is no code yet that allows different slots to be configured for storing certain types of entities.

Change 450069 had a related patch set uploaded (by Daniel Kinzler; owner: Daniel Kinzler):
[mediawiki/extensions/Wikibase@master] WIP MCR Allow alternative to be configured for entities

https://gerrit.wikimedia.org/r/450069

Change 442253 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Use PageUpdater & RevisionStore in Wikibase

https://gerrit.wikimedia.org/r/442253

daniel added a comment.Aug 5 2018, 6:53 PM

The change to WikiPageEntityRevisionLookup has been reverted, see T201194#4479699.

Change 450541 had a related patch set uploaded (by Daniel Kinzler; owner: Daniel Kinzler):
[mediawiki/extensions/Wikibase@master] Use PageUpdater & RevisionStore in Wikibase

https://gerrit.wikimedia.org/r/450541

daniel claimed this task.Aug 6 2018, 4:48 PM

Change 450541 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Use PageUpdater & RevisionStore in Wikibase

https://gerrit.wikimedia.org/r/450541

Change 450069 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Allow alternative slots to be configured for entities

https://gerrit.wikimedia.org/r/450069

daniel closed this task as Resolved.Aug 20 2018, 10:03 AM
Addshore moved this task from Watching to Done ✔️ on the User-Addshore board.Aug 20 2018, 3:17 PM

Change 461415 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/Wikibase@master] MCR: If a revision doesn't have our role, it doesn't have our entity

https://gerrit.wikimedia.org/r/461415

Change 461436 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/Wikibase@master] WIP Fix WikiPageEntityMetaDataLookup::getWhere for MCR

https://gerrit.wikimedia.org/r/461436

daniel reopened this task as Open.Fri, Sep 21, 12:31 PM
daniel reassigned this task from daniel to Addshore.

Re-opening, since issues were found. To be concrete, entities can be stored in extra slots, but Wikibase gets confused when finding a page that exists, could have an entity in an auxiliary slot, but doesn't. Adam is working on it.

Change 461415 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] MCR: If a revision doesn't have our role, it doesn't have our entity

https://gerrit.wikimedia.org/r/461415

Change 462929 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/Wikibase@master] MCR: Add SlotRoleStore to PerRepositoryServiceContainer

https://gerrit.wikimedia.org/r/462929

Change 463290 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/Wikibase@master] MCR: Create & Use TypeDispatchingWikiPageEntityMetaDataAccessor

https://gerrit.wikimedia.org/r/463290

Change 462929 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] MCR: Add SlotRoleStore to PerRepositoryServiceContainer

https://gerrit.wikimedia.org/r/462929

Change 463290 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] MCR: Create & Use TypeDispatchingWikiPageEntityMetaDataAccessor

https://gerrit.wikimedia.org/r/463290

All of the patches are now up, the final one of the chain currently being https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Wikibase/+/461967/

Change 463724 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/Wikibase@master] WIP DNM MCR: Introduce PageTableEntityConditionGenerator

https://gerrit.wikimedia.org/r/463724

Change 461967 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/Wikibase@master] MCR: Adjust how we work with EDIT_NEW/UPDATE for MCR

https://gerrit.wikimedia.org/r/461967

Change 463798 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/Wikibase@master] WIP DNM MCR: Fix EntitySavingHelper for MCR

https://gerrit.wikimedia.org/r/463798

Change 461436 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Fix WikiPageEntityMetaDataLookup::getWhere for MCR

https://gerrit.wikimedia.org/r/461436

Change 461967 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] MCR: Adjust how we work with EDIT_NEW/UPDATE for MCR

https://gerrit.wikimedia.org/r/461967

Change 463724 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] MCR: Introduce PageTableEntityQuery & Implementation

https://gerrit.wikimedia.org/r/463724