Page MenuHomePhabricator

Switch `tmpItemTermsMigrationStages` to MIGRATION_WRITE_NEW
Open, MediumPublic

Event Timeline

alaa_wmde set Due Date to Jul 2 2019, 10:00 PM.
Restricted Application changed the subtype of this task from "Task" to "Deadline". · View Herald TranscriptJun 5 2019, 6:55 AM
alaa_wmde renamed this task from Switch `tmpItemTermsMigrationStage` to MIGRATION_WRITE_NEW to Switch `tmpItemTermsTwoMillionMigrationStage` to MIGRATION_WRITE_NEW.Jun 5 2019, 8:00 AM
alaa_wmde renamed this task from Switch `tmpItemTermsTwoMillionMigrationStage` to MIGRATION_WRITE_NEW to Switch `tmpItemTermsMigrationStages` to MIGRATION_WRITE_NEW.Jun 13 2019, 5:58 PM
alaa_wmde changed Due Date from Jul 2 2019, 10:00 PM to Jul 9 2019, 10:00 PM.Jun 13 2019, 6:00 PM
Addshore moved this task from incoming to in progress on the Wikidata board.Jun 21 2019, 11:37 PM

Change 526643 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/mediawiki-config@master] labs: Set tmpItemTermsMigrationStages to MIGRATION_WRITE_BOTH for half of Wikidata

https://gerrit.wikimedia.org/r/526643

Change 526643 merged by jenkins-bot:
[operations/mediawiki-config@master] labs: Set tmpItemTermsMigrationStages to MIGRATION_WRITE_BOTH for half of Wikidata

https://gerrit.wikimedia.org/r/526643

Change 526643 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/mediawiki-config@master] labs: Set tmpItemTermsMigrationStages to MIGRATION_WRITE_BOTH for half of Wikidata
https://gerrit.wikimedia.org/r/526643

That supposed to go with T225055: Switch `tmpItemTermsMigrationStages` to MIGRATION_WRITE_BOTH instead of this ticket. Sorry

Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptNov 27 2019, 1:47 PM

Change 553342 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/mediawiki-config@master] Set all of testwikidatawiki to read from the new term store for items

https://gerrit.wikimedia.org/r/553342

Change 553342 merged by jenkins-bot:
[operations/mediawiki-config@master] Set all of testwikidatawiki to read from the new term store for items

https://gerrit.wikimedia.org/r/553342

Mentioned in SAL (#wikimedia-operations) [2019-11-27T13:57:57Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Set all of testwikidatawiki to read from the new term store for items (T225057) (duration: 00m 56s)

Test Wikidata is on read new now, we have time to check until next week that we enable reading new for wikidata (up to Q1000)

Change 554049 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/mediawiki-config@master] Set read new for term store for items of wikidata up to Q1000

https://gerrit.wikimedia.org/r/554049

Change 554049 merged by jenkins-bot:
[operations/mediawiki-config@master] Set read new for term store for items of wikidata up to Q1000

https://gerrit.wikimedia.org/r/554049

Mentioned in SAL (#wikimedia-operations) [2019-12-02T12:31:46Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:554049|Set read new for term store for items of wikidata up to Q1000 (T225057)]] (duration: 01m 00s)

Change 554164 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/mediawiki-config@master] Set item term store for read new up to Q1000 for clients

https://gerrit.wikimedia.org/r/554164

Change 554164 merged by jenkins-bot:
[operations/mediawiki-config@master] Set item term store for read new up to Q1000 for clients

https://gerrit.wikimedia.org/r/554164

Mentioned in SAL (#wikimedia-operations) [2019-12-03T12:09:31Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:554164|Set read new for term store for items for client wikis up to Q1000 (T225057)]] (duration: 01m 00s)

We moved to new store to up to Q1000. We wait for a while and move to a higher limit later

Addshore changed the task status from Open to Stalled.Jan 7 2020, 10:57 AM
Addshore added a subscriber: Addshore.

Stalled, waiting for T225055 and other moving parts

Addshore triaged this task as Medium priority.Jan 14 2020, 4:20 PM
Addshore changed the task status from Stalled to Open.Jan 15 2020, 5:16 PM
Addshore removed Ladsgroup as the assignee of this task.

Took your face off for now, but feel free to grba it back when ready

Change 565074 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/mediawiki-config@master] Set read for items in Wikidata for new term store up to Q8M

https://gerrit.wikimedia.org/r/565074

Change 565074 merged by jenkins-bot:
[operations/mediawiki-config@master] Set read for items in Wikidata for new term store up to Q8M

https://gerrit.wikimedia.org/r/565074

Mentioned in SAL (#wikimedia-operations) [2020-01-16T12:10:57Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:565074|Set read for items in Wikidata for new term store up to Q8M (T225057)]] (duration: 01m 07s)

  • We should not read terms from the new term store for item ids higher than where the migration script has run to (currently ~10,000,000), as past that point is not populated fully, hence nothing being able to lookup labels and falling back to item ids.
    • This started happening on wikis with the wmf.18 train rollout which included a patch changing the defaults in Wikibase to read from the new term store for all items https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Wikibase/+/566513/
    • This happened due to the oversight in configuration name which was changed in https://gerrit.wikimedia.org/r/570892, caused an outage, and then was reverted (NOT YET FIXED)
    • This fix meant that all wikis started reading from the new termstore for items under Q10,000,000 when previously the defaults were used meaning group0&1 reading from the new term store for all items and group2 was reading from the new termstore for no items.

Current state:

  • group0 and group1 sites will retrieve no or incorrect data when looking up most item labels and descriptions. Lots of these lookups will end up causing unexpected alterations in parsed and cached pages.
  • group2 is currently on .16 so still reading the wikibase default setting from .16, which is, do not read from the new terms storage for an item ids.

UBNs:

This will leave the train able to continue and all client wikis reading from the old term store (as everything was on monday of this week before .18 rolled out).
Folowups:

  • Alter mediawiki-config wmgWikibaseTmpItemTermsMigrationStage values in mediawiki-config to match what is currently being used in production for clients (read from old everywhere)
  • Fix the incorrectly named config var in mediawiki-config
  • Move forward with changes reading from the new term store for items on clients (see T225057: Switch `tmpItemTermsMigrationStages` to MIGRATION_WRITE_NEW) in a controlled manner.