Page MenuHomePhabricator
Create Task
Maniphest T224541

Update Wikibase entity saving logic to use PropertyTermStore
Closed, ResolvedPublic
Actions

Description

PropertyTermStore should be used when migration is switched to write new mode.

Details

SubjectRepoBranchLines +/-
Add MultiItemTermStoremediawiki/extensions/Wikibasemaster+266 -0
Add TermIndexItemTermStore adaptermediawiki/extensions/Wikibasemaster+198 -0
Wire up DatabasePropertyTermStore in WikibaseRepomediawiki/extensions/Wikibasemaster+69 -5
Add MultiPropertyTermStoremediawiki/extensions/Wikibasemaster+266 -0
Add TermIndexPropertyTermStore adaptermediawiki/extensions/Wikibasemaster+198 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

• alaa_wmde created this task.May 29 2019, 8:02 AM
• alaa_wmde moved this task from Backlog to Ready to go [prioritized] on the Wikidata wb_terms Trailblazing board.May 29 2019, 8:14 AM
Lucas_Werkmeister_WMDE edited parent tasks, added: T219121: [Checkpoint 2] Refactor Write & Migration code for Properties; removed: T219122: [Checkpoint 4] Refactor Write & Migration code for Items.May 29 2019, 11:54 AM
Lucas_Werkmeister_WMDE claimed this task.May 29 2019, 11:57 AM
Lucas_Werkmeister_WMDE moved this task from Ready to go [prioritized] to In Progress on the Wikidata wb_terms Trailblazing board.
Lucas_Werkmeister_WMDE subscribed.

Currently, we have this in WikibaseRepo:

	private function newEntityTermStoreWriter(): EntityTermStoreWriter {
		$termStoreMigrationStage = $this->settings->getSetting( 'tmpTermStoreMigrationStage' );

		if ( $termStoreMigrationStage === MIGRATION_WRITE_BOTH ) {
			return new MultiTermStoreWriter(
				$this->getOldEntityTermStoreWriter(),
				$this->getNewEntityTermStoreWriter()
			);
		}

		if ( $termStoreMigrationStage === MIGRATION_WRITE_NEW ) {
			return $this->getNewEntityTermStoreWriter();
		}

		return $this->getOldEntityTermStoreWriter();
	}

	private function getOldEntityTermStoreWriter(): EntityTermStoreWriter {
		return $this->getStore()->getTermIndex();
	}

	private function getNewEntityTermStoreWriter(): EntityTermStoreWriter {
		return new DelegatingEntityTermStoreWriter(
			$this->getPropertyTermStore(),
			$this->getItemTermStore()
		);
	}

I can only assume that this predates the current migration plan, because it doesn’t really fit it very well: it assumes that we’ll do a migration across all entities at the same time, whereas we want to do properties first, and then items only up to a certain numeric ID.

gerritbot added a comment.May 29 2019, 1:34 PM

Change 513118 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/Wikibase@master] Add TermIndexPropertyTermStore adapter

https://gerrit.wikimedia.org/r/513118

gerritbot added a project: Patch-For-Review.May 29 2019, 1:34 PM
Lucas_Werkmeister_WMDE moved this task from In Progress to In Review on the Wikidata wb_terms Trailblazing board.May 29 2019, 1:34 PM
gerritbot added a comment.May 29 2019, 3:31 PM

Change 513144 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/Wikibase@master] Add MultiPropertyTermStore

https://gerrit.wikimedia.org/r/513144

gerritbot added a comment.May 29 2019, 4:54 PM

Change 513158 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/Wikibase@master] Wire up DatabasePropertyTermStore in WikibaseRepo

https://gerrit.wikimedia.org/r/513158

• alaa_wmde added a comment.May 31 2019, 9:23 AM
In T224541#5220395, @Lucas_Werkmeister_WMDE wrote:

Currently, we have this in WikibaseRepo:

	private function newEntityTermStoreWriter(): EntityTermStoreWriter {
		$termStoreMigrationStage = $this->settings->getSetting( 'tmpTermStoreMigrationStage' );

		if ( $termStoreMigrationStage === MIGRATION_WRITE_BOTH ) {
			return new MultiTermStoreWriter(
				$this->getOldEntityTermStoreWriter(),
				$this->getNewEntityTermStoreWriter()
			);
		}

		if ( $termStoreMigrationStage === MIGRATION_WRITE_NEW ) {
			return $this->getNewEntityTermStoreWriter();
		}

		return $this->getOldEntityTermStoreWriter();
	}

	private function getOldEntityTermStoreWriter(): EntityTermStoreWriter {
		return $this->getStore()->getTermIndex();
	}

	private function getNewEntityTermStoreWriter(): EntityTermStoreWriter {
		return new DelegatingEntityTermStoreWriter(
			$this->getPropertyTermStore(),
			$this->getItemTermStore()
		);
	}

I can only assume that this predates the current migration plan, because it doesn’t really fit it very well: it assumes that we’ll do a migration across all entities at the same time, whereas we want to do properties first, and then items only up to a certain numeric ID.

Yeap that was introduced too early, without much attention to the actual migration plan. (I do remember rejecting that change, but maybe I'm confusing it with something else).

gerritbot added a comment.May 31 2019, 11:13 AM

Change 513118 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Add TermIndexPropertyTermStore adapter

https://gerrit.wikimedia.org/r/513118

gerritbot added a comment.May 31 2019, 11:13 AM

Change 513144 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Add MultiPropertyTermStore

https://gerrit.wikimedia.org/r/513144

ReleaseTaggerBot added a project: MW-1.34-notes (1.34.0-wmf.8; 2019-06-04).May 31 2019, 12:00 PM
gerritbot added a comment.Jun 3 2019, 1:33 PM

Change 513158 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Wire up DatabasePropertyTermStore in WikibaseRepo

https://gerrit.wikimedia.org/r/513158

Maintenance_bot removed a project: Patch-For-Review.Jun 3 2019, 2:11 PM
Lucas_Werkmeister_WMDE closed this task as Resolved.Jun 3 2019, 4:23 PM
Lucas_Werkmeister_WMDE moved this task from In Review to Done on the Wikidata wb_terms Trailblazing board.
gerritbot added a comment.Jun 3 2019, 5:44 PM

Change 514078 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/Wikibase@master] Add TermIndexItemTermStore adapter

https://gerrit.wikimedia.org/r/514078

gerritbot added a project: Patch-For-Review.Jun 3 2019, 5:44 PM

Change 514079 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/Wikibase@master] Add MultiItemTermStore

https://gerrit.wikimedia.org/r/514079

gerritbot added a comment.Jun 4 2019, 9:59 AM

Change 514078 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Add TermIndexItemTermStore adapter

https://gerrit.wikimedia.org/r/514078

gerritbot added a comment.Jun 4 2019, 1:34 PM

Change 514079 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Add MultiItemTermStore

https://gerrit.wikimedia.org/r/514079

ReleaseTaggerBot edited projects, added MW-1.34-notes (1.34.0-wmf.10; 2019-06-18); removed MW-1.34-notes (1.34.0-wmf.8; 2019-06-04).Jun 4 2019, 2:00 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits