Page MenuHomePhabricator

mw.wikibase.getLabelByLang not return item label for some items, because MIGRATION_WRITE_NEW is being deployed with train for wmf.18
Closed, ResolvedPublic

Description

  1. Go to https://www.wikidata.org/wiki/Wikidata:WikiProject_Linked_Data_for_Production/Finding_GLAMs_Campaign#Test with Firefox version 72.0.2 or Safari 13.0.3.

Actual results:
Q77246866
Q77246869
Universe (Q1)

Expected results:
Stanford University Aeronautics & Astronautics Department (Q77246866)
Stanford University Anesthesia Department (Q77246869)
Universe (Q1)

Wikicode:
{{Q|77246866}}

{{Q|77246869}}

{{Q|1}}

It seems to be an issue with items that have been created recently. I asked about it on the Wikidata Telegram channel and someone thought it looked like an internal problem with "mw.wikibase.getLabelByLang"

Details

Related Gerrit Patches:
mediawiki/extensions/Wikibase : masterRevert "wbterms: Set default for the term store to read new"
mediawiki/extensions/Wikibase : wmf/1.35.0-wmf.18Revert "wbterms: Set default for the term store to read new"
operations/mediawiki-config : masterWikibase Client: Fix setting name typo

Event Timeline

Restricted Application added subscribers: Liuxinyu970226, Aklapper. · View Herald TranscriptThu, Feb 6, 7:50 PM
Jarekt renamed this task from {{Q|id}} syntax not displaying item label sometimes to mw.wikibase.getLabelByLang not return item label for some items.Fri, Feb 7, 4:20 AM
Jarekt added a project: Wikibase-Lua.
Jarekt added a subscriber: Jarekt.Fri, Feb 7, 4:32 AM

As an author of Lua code behind {{Q|}} template and few others related ones I was alerted about this issue. I tried the following test; I added labelByLang function to c:Module:Test:

local p = {}

function p.labelByLang(frame)
	label = mw.wikibase.getLabelByLang( frame.args[1], frame.args[2] )
	return label or 'None'
end

return p

Than tested it by:

  • {{#invoke:Test|labelByLang|Q75820373|en}} gives "None" (expected "WikiProject Supertemplates", for Q75820373)
  • {{#invoke:Test|labelByLang|Q1|en}} gives "Universe" (correct)

I got similar error for mw.wikibase.getLabelAndLang function

Jarekt added a subscriber: hoo.Fri, Feb 7, 4:38 AM

Adding hoo (Marius Hoch) who wrote mw.wikibase.getLabelByLang function in the first place.

Bugreporter triaged this task as Unbreak Now! priority.EditedFri, Feb 7, 7:35 AM
Bugreporter added a subscriber: Bugreporter.

See also https://www.wikidata.org/wiki/Wikidata:Project_chat#Missing_labels_again, items higher than Q74000000 are not in new term store at all and are broken. This means Wikibase Lua library reads only new term store, not old one.

matej_suchanek added a subscriber: matej_suchanek.

Not Lua specific, #property and #statements have the same problem, eg. {{#property:P527|from=Q5836034}}. Reproduced on Wikidata and Commons but not enwiki.

Lydia_Pintscher added a subscriber: Lydia_Pintscher.

😱
Bringing it up with the team in a sec.

Addshore added a subscriber: Addshore.

Probably not.
Items should only be reading up to around 8 million currently for the new terms store.

But it is likely related to the migration in general.

If it happens on Q74000000 and higher it's probably because the code decides to read from the new term store while it shouldn't. In debugging, I would start with this.

hoo claimed this task.Fri, Feb 7, 1:17 PM
hoo added a comment.EditedFri, Feb 7, 1:40 PM

Interesting:

hoo@mwmaint1002:~$ mwscript eval.php --wiki wikidatawiki

> $wbRepo = Wikibase\Repo\WikibaseRepo::getDefaultInstance();

> $wbClient = Wikibase\Client\WikibaseClient::getDefaultInstance();

> $clientTermLookup = $wbClient->getTermLookup();

> $repoTermLookup = $wbRepo->getPrefetchingTermLookup();

> $item = new Wikibase\DataModel\Entity\ItemId( 'Q77246866' );

> echo $clientTermLookup->getLabel( $item, 'en' );

> echo $repoTermLookup->getLabel( $item, 'en' );
Stanford University Aeronautics & Astronautics Department

Change 570892 had a related patch set uploaded (by Hoo man; owner: Hoo man):
[operations/mediawiki-config@master] Wikibase Client: Fix setting name typo

https://gerrit.wikimedia.org/r/570892

Change 570892 merged by jenkins-bot:
[operations/mediawiki-config@master] Wikibase Client: Fix setting name typo

https://gerrit.wikimedia.org/r/570892

Mentioned in SAL (#wikimedia-operations) [2020-02-07T14:38:02Z] <hoo@deploy1001> Synchronized wmf-config/Wikibase.php: Wikibase Client: Fix setting name typo (T244529) (duration: 01m 20s)

Mentioned in SAL (#wikimedia-operations) [2020-02-07T14:43:43Z] <hoo@deploy1001> Synchronized wmf-config/Wikibase.php: REVERT: Wikibase Client: Fix setting name typo (T244529) (duration: 01m 40s)

hoo added a comment.Fri, Feb 7, 2:47 PM

We don't know what happened, but the fix to this caused cluster wide timeouts, maybe the new term storage couldn't yet handle the additional load? Will need to be investigated and we need another short-term fix.

Tarrow added a subscriber: Tarrow.Fri, Feb 7, 2:52 PM
Addshore added a comment.EditedFri, Feb 7, 3:33 PM

The repo config had this same typo part way through 2019 but it was fixed there, https://gerrit.wikimedia.org/r/#/c/526646

Change 570910 had a related patch set uploaded (by Tarrow; owner: Tarrow):
[mediawiki/extensions/Wikibase@master] Revert "wbterms: Set default for the term store to read new"

https://gerrit.wikimedia.org/r/570910

Change 570911 had a related patch set uploaded (by Tarrow; owner: Tarrow):
[mediawiki/extensions/Wikibase@wmf/1.35.0-wmf.18] Revert "wbterms: Set default for the term store to read new"

https://gerrit.wikimedia.org/r/570911

Addshore renamed this task from mw.wikibase.getLabelByLang not return item label for some items to mw.wikibase.getLabelByLang not return item label for some items, because MIGRATION_WRITE_NEW is being deployed with train for wmf.18.Fri, Feb 7, 3:57 PM
Addshore added a comment.EditedFri, Feb 7, 4:10 PM
  • We should not read terms from the new term store for item ids higher than where the migration script has run to (currently ~10,000,000), as past that point is not populated fully, hence nothing being able to lookup labels and falling back to item ids.
    • This started happening on wikis with the wmf.18 train rollout which included a patch changing the defaults in Wikibase to read from the new term store for all items https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Wikibase/+/566513/
    • This happened due to the oversight in configuration name which was changed in https://gerrit.wikimedia.org/r/570892, caused an outage, and then was reverted (NOT YET FIXED)
    • This fix meant that all wikis started reading from the new termstore for items under Q10,000,000 when previously the defaults were used meaning group0&1 reading from the new term store for all items and group2 was reading from the new termstore for no items.

Current state:

  • group0 and group1 sites will retrieve no or incorrect data when looking up most item labels and descriptions. Lots of these lookups will end up causing unexpected alterations in parsed and cached pages.
  • group2 is currently on .16 so still reading the wikibase default setting from .16, which is, do not read from the new terms storage for an item ids.

UBNs:

This will leave the train able to continue and all client wikis reading from the old term store (as everything was on monday of this week before .18 rolled out).

Folowups:

  • Alter mediawiki-config wmgWikibaseTmpItemTermsMigrationStage values in mediawiki-config to match what is currently being used in production for clients (read from old everywhere)
  • Fix the incorrectly named config var in mediawiki-config
  • Move forward with changes reading from the new term store for items on clients (see T225057: Switch `tmpItemTermsMigrationStages` to MIGRATION_WRITE_NEW) in a controlled manner.
Michael added a subscriber: Michael.Fri, Feb 7, 4:52 PM

Change 570910 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Revert "wbterms: Set default for the term store to read new"

https://gerrit.wikimedia.org/r/570910

Tarrow added a comment.Fri, Feb 7, 5:21 PM

We're rolling back to wmf.16 on all wikis. We hope this will resolve the issues but of course some bad data may now be cached. We'll look at this again on Monday.

We're rolling back to wmf.16 on all wikis. We hope this will resolve the issues but of course some bad data may now be cached. We'll look at this again on Monday.

Thanks - caches need purging, but otherwise things seem to be working OK again now.

Leaving this as UBN for now as it still blocks the .18 train (if that train is to continue / restart) per what I said in T244529#5859927.
I'm away next week, @Tarrow knows everything I know.
If the next train to roll is .19 this will not be an issue as the revert for the wikibase default is already merged on master. (in which case this can be downgraded from UBN)
Followups still need to be done

Tarrow claimed this task.Sat, Feb 8, 5:23 PM

Change 570911 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@wmf/1.35.0-wmf.18] Revert "wbterms: Set default for the term store to read new"

https://gerrit.wikimedia.org/r/570911

Mentioned in SAL (#wikimedia-operations) [2020-02-10T13:03:09Z] <ladsgroup@deploy1001> Synchronized php-1.35.0-wmf.18/extensions/Wikibase: [[gerrit:570911|Revert "wbterms: Set default for the term store to read new"]] (T244529) (duration: 01m 00s)

There's nothing to review for now, we are waiting for deployment of wmf.18 to see if it works as intended or not.

Ladsgroup closed this task as Resolved.Mon, Feb 10, 7:20 PM

This seems to be fixed now. The AC passes now.

The only config change linked to this ticket is https://gerrit.wikimedia.org/r/#/c/570892 which is the one that was reverted.
I'm guessing another one was also merged?

I'm guessing this was:

12:28 ladsgroup@deploy1001: Synchronized wmf-config/Wikibase.php: SWAT: Fix typo in the config name (T244697), take II, cache (duration: 01m 06s)
12:26 ladsgroup@deploy1001: Synchronized wmf-config/Wikibase.php: SWAT: Fix typo in the config name (T244697) (duration: 01m 05s)
12:12 ladsgroup@deploy1001: Synchronized wmf-config/InitialiseSettings.php: SWAT: Stop reading for the new term store as the default of client wikis (T244697), Second round, cache issue (duration: 01m 07s)
12:10 ladsgroup@deploy1001: Synchronized wmf-config/InitialiseSettings.php: SWAT: Stop reading for the new term store as the default of client wikis (T244697) (duration: 01m 11s)

on 11th Feb and thus...

https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/571338/
https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/571339/

It looks like T244697 was created but not linked to this ticket.