[Regression wmf.13] Wikidata localisation is broken
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Assigned To

Authored By

	Mormegil
	Jul 11 2019, 7:42 PM

Description

Wikidata localisation seems to be broken quite a lot; when viewing in (e.g.) Czech (as always), I notice many broken things everywhere (e.g. a completely random example):

the sitelinks box is captioned “⧼wikibase-sitelinks-wikipedia⧽”
date statements show “http://www.wikidata.org/entity/Q1985727” instead of the calendar name
history displays things like “wbcreateclaim-create:1|” instead of the human-readable text

The same happens when I use e.g. uselang=de; on the other hand, the page looks fine with uselang=en.

Current status (2019-07-11Z20:57): Wikidata rolled back to wmf.11. Breakage visible on Test Wikidata, e.g. https://test.wikidata.org/wiki/Q64507

Details

	Subject	Repo	Branch	Lines +/-
	Define $wgMessagesDirs in WikibaseLib PHP entry point	mediawiki/extensions/Wikibase	wmf/1.34.0-wmf.14	+6 -0
	Define $wgMessagesDirs in WikibaseLib PHP entry point	mediawiki/extensions/Wikibase	master	+6 -0

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved	Release	jeena	T220738 1.34.0-wmf.13 deployment blockers
		Resolved	PRODUCTION ERROR	greg	T227814 [Regression wmf.13] Wikidata localisation is broken

Event Timeline

Mormegil created this task.Jul 11 2019, 7:42 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 11 2019, 7:42 PM

Mormegil mentioned this in T227815: Message wikibase-sitelinks-wikipedia missing from wikidata.org.Jul 11 2019, 7:44 PM

Jdforrester-WMF renamed this task from Wikidata localization is broken to [Regression wmf.13] Wikidata localisation is broken.Jul 11 2019, 8:00 PM

Jdforrester-WMF triaged this task as High priority.

Jdforrester-WMF merged a task: T227815: Message wikibase-sitelinks-wikipedia missing from wikidata.org.

Jdforrester-WMF added projects: Performance-Team (Radar), Wikimedia-production-error (ARCHIVED -- Shared Build Failure), Release-Engineering-Team-TODO (201907).

Jdforrester-WMF updated the task description. (Show Details)

Jdforrester-WMF added subscribers: Jdforrester-WMF, Krinkle.

Hrm. That is bizarre.

$ grep -c wikibase-sitelinks-wikipedia /srv/mediawiki-staging/php-1.34.0-wmf.13/cache/l10n/upstream/l10n_cache-cs.cdb.json
0
$ grep -c wikibase-sitelinks-wikipedia /srv/mediawiki-staging/php-1.34.0-wmf.11/cache/l10n/upstream/l10n_cache-cs.cdb.json
2

Nothing looks to have changed with the actual json in the extension:

https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/Wikibase/+/wmf/1.34.0-wmf.11/lib/i18n/cs.json#53

https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/Wikibase/+/wmf/1.34.0-wmf.13/lib/i18n/cs.json#53

So either scap made a bad change, something scap shelled out to made a bad change, or there was some runtime error, methinks.

Krinkle added a parent task: T220738: 1.34.0-wmf.13 deployment blockers.Jul 11 2019, 8:18 PM

UBN since it's a deployment blocker.

It's not in the cdb file either

$ strings /srv/mediawiki-staging/php-1.34.0-wmf.13/cache/l10n//l10n_cache-cs.cdb | grep -c 'wikibase-sitelinks-wikipedia'
0
$ strings /srv/mediawiki-staging/php-1.34.0-wmf.11/cache/l10n/l10n_cache-cs.cdb | grep -c 'wikibase-sitelinks-wikipedia'
2

So it's not an error in the scap logic that generates json from cdb files, but in the logic/runtime that generates the cdbs.

Restricted Application added a subscriber: Liuxinyu970226. · View Herald TranscriptJul 11 2019, 8:27 PM

Change 522181 had a related patch set uploaded (by Jeena Huneidi; owner: Jeena Huneidi):
[operations/mediawiki-config@master] Revert wikidata to 1.34.0-wmf.11 refs T227814

https://gerrit.wikimedia.org/r/522181

gerritbot added a project: Patch-For-Review.Jul 11 2019, 8:35 PM

Change 522181 merged by jenkins-bot:
[operations/mediawiki-config@master] Revert wikidata to 1.34.0-wmf.11 refs T227814

https://gerrit.wikimedia.org/r/522181

hrm, so WikibaseLib is in ExtensionMessages for 1.34.11, but not in ExtensionsMessages for 1.34.13

$ diff wmf-config/ExtensionMessages-1.34.0-wmf.{11,13}.php
...
918,925d922
<   'WikibaseLib' =>
<   array (
<     0 => "$IP/extensions/Wikibase/lib/../lib/i18n",
<   ),
<   'WikibaseView' =>
<   array (
<     0 => "$IP/extensions/Wikibase/view/../view/lib/wikibase-data-values-value-view/i18n",                                                           
<   ),
...

This file is generated by mergeMessageFileLists.php.

In the scap logs I see:

2019-07-09T19:44:11 Loading data from /srv/mediawiki-staging/php-1.34.0-wmf.13/extensions/Wikibase/client/WikibaseClient.php
2019-07-09T19:44:11 Loading data from /srv/mediawiki-staging/php-1.34.0-wmf.13/extensions/Wikibase/repo/Wikibase.php

The 2 previous week's output looks the same:

2019-06-25T19:51:39 Loading data from /srv/mediawiki-staging/php-1.34.0-wmf.11/extensions/Wikibase/client/WikibaseClient.php
2019-06-25T19:51:39 Loading data from /srv/mediawiki-staging/php-1.34.0-wmf.11/extensions/Wikibase/repo/Wikibase.php

Jdforrester-WMF updated the task description. (Show Details)Jul 11 2019, 8:57 PM

Nothing obvious in the repo's logs.

Maintenance_bot removed a project: Patch-For-Review.Jul 11 2019, 9:10 PM

Tacsipacsi subscribed.Jul 11 2019, 9:25 PM

In T227814#5326488, @Jdforrester-WMF wrote:

Nothing obvious in the repo's logs.

I found https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Wikibase/+/515031/ which looks suspect and is fairly recent; however, that should have affected 1.34.0-wmf.11 as well, I think.

Yeah, I remembered that one but as you say it went out in wmf.10.

[[ https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/core/+/master/maintenance/mergeMessageFileList.php | mergeMessageFileList ]] which is what scap shells out to to handle building ExtensionMessages does a require_once on each of the files listed in the extension-list file. It then dumps $wgMessagesDirs into the ExtensionMessages file.

This seems somewhat straightforward; however, maybe there's a possible race-condition here somewhere?

Mentioned in SAL (#wikimedia-operations) [2019-07-11T22:07:16Z] <thcipriani@deploy1001> Started scap: no op scap sync to rebuild l10n-cache (T227814)

Ran in another terminal window after I got the message 22:10:59 Updating ExtensionMessages-1.34.0-wmf.13.php from scap

$ grep -i wikibaselib wmf-config/ExtensionMessages-1.34.0-wmf.13.php 
$ grep -i wikibaselib wmf-config/ExtensionMessages-1.34.0-wmf.13.php 
  'WikibaseLib' =>

Mentioned in SAL (#wikimedia-operations) [2019-07-11T22:26:50Z] <thcipriani@deploy1001> Finished scap: no op scap sync to rebuild l10n-cache (T227814) (duration: 19m 34s)

Seems to be fixed.

Thanks and sorry all.

Restricted Application added a project: User-greg. · View Herald TranscriptJul 12 2019, 2:21 AM

Really fixed? Testwikidata is still bad for me, the monolingual text statement’s content for example is not visible at all.

Lea_Lacroix_WMDE subscribed.Jul 12 2019, 10:00 AM

So did we figure out what happened? (The change @thcipriani linked is by me, and I also think it feels suspect, so I hope this is not my fault 😬)

Edit: And if this was only fixed by reverting the train, and the issue persists on Test Wikidata, shouldn’t the task remain open?

Test Wikidata works fine for me, the broken display disappeared on purge.

Michael subscribed.Jul 12 2019, 2:47 PM

In T227814#5327808, @Lucas_Werkmeister_WMDE wrote:

So did we figure out what happened? (The change @thcipriani linked is by me, and I also think it feels suspect, so I hope this is not my fault 😬)

Edit: And if this was only fixed by reverting the train, and the issue persists on Test Wikidata, shouldn’t the task remain open?

1.34.0-wmf.13 is on all wikis, this was fixed by re-running scap sync.

I'm not exactly happy about that outcome since I don't understand how it failed the first go-round. I looked closely at mergeMessageFileList.php today (which is the file that generates ExtensionsMessages-[version].php) and I'm not clear on how it works at all.

Essentially what mergeMessageFileList.php seems to do is do a require_once for all php files in extension-list (pre-seeding the $wgExtensionMessagesFiles and $wgMessagesDirs variables), then it extracts the global variables wgExtensionMessagesFiles and wgMessagesDirs from any json file in extension-list; however, when a file you've required adds a json file to the ExtensionRegistration queue (as in the Wikibase patch) when is that queue processed? I don't know, but it seemingly does (at least most of the time).

In T227814#5327808, @Lucas_Werkmeister_WMDE wrote:

So did we figure out what happened? (The change @thcipriani linked is by me, and I also think it feels suspect, so I hope this is not my fault 😬)

Looking at https://www.mediawiki.org/wiki/Manual:Extension_registration#Migration_for_extension_developers it looks like you should keep $wgMessagesDirs specifically for mergeMessageFileList (see the comment in the example code on that page)

<?php
if ( function_exists( 'wfLoadExtension' ) ) {
	wfLoadExtension( 'FooBar' );
	// Keep i18n globals so mergeMessageFileList.php doesn't break
	$wgMessagesDirs['FooBar'] = __DIR__ . '/i18n';

Another example is Charlie Barnes (Q21005322):

Screenshot 2019-07-14 at 11.44.18.png (590×937 px, 117 KB)

Premeditated reopened this task as Open.Jul 14 2019, 9:48 AM

Once again, this solved itself by purging the page, probably just a cached version of the broken state.

In T227814#5329364, @thcipriani wrote:
Looking at https://www.mediawiki.org/wiki/Manual:Extension_registration#Migration_for_extension_developers it looks like you should keep $wgMessagesDirs specifically for mergeMessageFileList (see the comment in the example code on that page)
<?php
if ( function_exists( 'wfLoadExtension' ) ) {
	wfLoadExtension( 'FooBar' );
	// Keep i18n globals so mergeMessageFileList.php doesn't break
	$wgMessagesDirs['FooBar'] = __DIR__ . '/i18n';

I asked about that on the talk page and the conclusion was that this is only necessary for extension-list entries, so removing the i18n globals in the WikibaseLib entry point should have been fine. If mergeMessageFileList.php does a require_once of the WikibaseRepo or -Client entry points, then it would also end up with the WikibaseLib entry point, and perhaps in that case we would need to keep the i18n globals after all… but it seems like, at least sometimes, having them only in the JSON file works? No idea why, though.

I think we should try to finish the Wikibase move to extension registration soon, and we’re actively working on it, but until we’re done with that perhaps it’s better to add the i18n globals back to the PHP entry points just to be sure?

Anyways, closing this task again since it seems like it’s still fixed, broken pages just need to be purged.

Lucas_Werkmeister_WMDE mentioned this in T228520: Wikibase: messages displayed instead of parsed.Jul 19 2019, 2:46 PM

Change 524539 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/Wikibase@master] Define $wgMessagesDirs in WikibaseLib PHP entry point

https://gerrit.wikimedia.org/r/524539

gerritbot added a project: Patch-For-Review.Jul 19 2019, 3:17 PM

Change 524539 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Define $wgMessagesDirs in WikibaseLib PHP entry point

https://gerrit.wikimedia.org/r/524539

ReleaseTaggerBot added a project: MW-1.34-notes (1.34.0-wmf.15; 2019-07-23).Jul 19 2019, 6:00 PM

Change 524573 had a related patch set uploaded (by Jforrester; owner: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/Wikibase@wmf/1.34.0-wmf.14] Define $wgMessagesDirs in WikibaseLib PHP entry point

https://gerrit.wikimedia.org/r/524573

Change 524573 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@wmf/1.34.0-wmf.14] Define $wgMessagesDirs in WikibaseLib PHP entry point

https://gerrit.wikimedia.org/r/524573

Mentioned in SAL (#wikimedia-operations) [2019-07-22T15:35:12Z] <jforrester@deploy1001> Synchronized php-1.34.0-wmf.14/extensions/Wikibase/lib/WikibaseLib.php: T227814 Wikibase: Define $wgMessagesDirs in WikibaseLib PHP entry point (duration: 00m 48s)

ReleaseTaggerBot edited projects, added MW-1.34-notes (1.34.0-wmf.14; 2019-07-16); removed MW-1.34-notes (1.34.0-wmf.15; 2019-07-23).Jul 22 2019, 4:00 PM

Maintenance_bot removed a project: Patch-For-Review.Jul 22 2019, 4:10 PM

brennen subscribed.Aug 2 2019, 3:59 PM

• mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:05 PM

	F29766318: Screenshot 2019-07-14 at 11.44.08.png
	Jul 14 2019, 9:46 AM

	F29766321: Screenshot 2019-07-14 at 11.44.18.png
	Jul 14 2019, 9:46 AM

[Regression wmf.13] Wikidata localisation is brokenClosed, ResolvedPublicPRODUCTION ERRORActions

Description

Details

Related ObjectsSearch...

Event Timeline

[Regression wmf.13] Wikidata localisation is broken
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Related Objects
Search...