Page MenuHomePhabricator

Interwiki prefix "wikipedia" not working on multilingual wikimedia projects
Closed, ResolvedPublicBUG REPORT

Description

Interwiki prefix "wikipedia" not working on multilingual wikimedia projects (All?). Type "[[wikipedia:test]]" on the page, and it will only be displayed as a red link after saving.

A page where this error occurred: https://commons.wikimedia.org/wiki/MediaWiki:Exif-fnumber

Test case:

Event Timeline

Shizhao renamed this task from Interwiki link "wikipedia" not working to Interwiki prefix "wikipedia" not working.Feb 17 2022, 3:16 AM
Shizhao updated the task description. (Show Details)
Shizhao renamed this task from Interwiki prefix "wikipedia" not working to Interwiki prefix "wikipedia" not working on multilingual wikimedia projects .Feb 17 2022, 3:25 AM
Shizhao updated the task description. (Show Details)

The "wikipedia" interwiki prefix still exists according to the API, same as "w": https://commons.wikimedia.org/wiki/Special:ApiSandbox#action=query&format=json&meta=siteinfo&formatversion=2&siprop=interwikimap

            {
                "prefix": "wikipedia",
                "local": true,
                "url": "https://en.wikipedia.org/wiki/$1",
                "protorel": false
            },
...
            {
                "prefix": "w",
                "local": true,
                "url": "https://en.wikipedia.org/wiki/$1",
                "protorel": false
            },

But trying to parse a title like "wikipedia:test" says it's a normal page in main namespace, while "w:test" is an interwiki: https://commons.wikimedia.org/wiki/Special:ApiSandbox#action=query&format=json&prop=&meta=&titles=wikipedia%3Atest%7Cw%3Atest&formatversion=2

"query": {
    "normalized": [
        {
            "fromencoded": false,
            "from": "wikipedia:test",
            "to": "Wikipedia:test"
        }
    ],
    "interwiki": [
        {
            "title": "w:test",
            "iw": "w"
        }
    ],
    "pages": [
        {
            "ns": 0,
            "title": "Wikipedia:test",
            "missing": true
        }
    ]
}

on enwikisource: https://en.wikisource.org/wiki/Special:ApiSandbox#action=query&format=json&prop=&meta=&titles=wikipedia%3Atest%7Cw%3Atest&formatversion=2

{
    "batchcomplete": true,
    "query": {
        "interwiki": [
            {
                "title": "wikipedia:test",
                "iw": "wikipedia"
            },
            {
                "title": "w:test",
                "iw": "w"
            }
        ]
    }
}

@matmarex

Not sure if its related but b90d2dd5c27a0db7a8a9ae14ffa1b2024f5c8678 relates to the main namespace processing

Should this be added as a train blocker?

taavi triaged this task as Unbreak Now! priority.Feb 17 2022, 8:32 AM
taavi added a subscriber: taavi.

Train blocker => UBN! by definition.

The mystery here is why only the wikipedia: interwiki appears to be affected. If this is indeed the case, it's not actually a blocker to deploying to group2 (Wikipedia sites don't use the wikipedia: interwiki). But it's a bad regression on the group0/group1 sites currently impacted, including big ones like Commons, Meta and Wikidata.

hashar added a subscriber: hashar.

Essentially wmf.22 broke the wikipedia: interwikis or maybe it is due to the interwiki configuration being broken. To be investigated. For now I am keeping wmf.22 deployed everywhere.

We have the list of interwiki via a special page https://www.mediawiki.org/wiki/Special:Interwiki or https://meta.wikimedia.org/wiki/Special:Interwiki both show:

PrefixURLForwardTransclude
wikipediahttps://en.wikipedia.org/wiki/$1yes-

On my local instance the external interwiki seems to be properly recognized:

$ echo '[[wikipedia:Foo]]'|php maintenance/parse.php
<p><a href="https://en.wikipedia.org/wiki/Foo" class="extiw" title="wikipedia:Foo">wikipedia:Foo</a>
</p>

In operations/mediawiki-config the cache wmf-config/interwiki.php has:

	'__global:wikipedia' => '1 https://en.wikipedia.org/wiki/$1',
...
	'_wiki:wikipedia' => '',

And those entries have not changed for a while. Same in production:

$ mwscript extensions/WikimediaMaintenance/dumpInterwiki.php --wiki=metawiki|grep ':wikipedia\b'
	'__global:wikipedia' => '1 https://en.wikipedia.org/wiki/$1',
	'_wiki:wikipedia' => '',

I will look at reverting https://gerrit.wikimedia.org/r/c/mediawiki/core/+/760695 on the beta cluster and see whether that fixes it.

I have disabled the Jenkins agent https://integration.wikimedia.org/ci/computer/deployment%2Ddeploy03/ to prevent automatic code update on the cluster. On the host I have reverted three patches that affected includes/skins/Skin.php:

cd /srv/mediawiki-staging/php-master
sudo -u jenkins-deploy git revert f7693089a8a
sudo -u jenkins-deploy git revert a516f69399d
sudo -u jenkins-deploy git revert b90d2dd5c27
scap sync-world

I then went to https://meta.wikimedia.beta.wmflabs.org/wiki/T301936 and previewed an edit for [[wikipedia:Foobar]], the interwiki was rendered properly.

I then dropped the patch b90d2dd5c27 Optimise Skin::getLanguages() (9 days ago) <Tim Starling> ran sync-world and the preview had the wikipedia: interwiki prefix broken again.

The root cause is thus the patch @matmarex identified earlier: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/760695

I will revert it.

Change 763488 had a related patch set uploaded (by Hashar; author: Hashar):

[mediawiki/core@master] Revert \"Optimise Skin::getLanguages()\"

https://gerrit.wikimedia.org/r/763488

Change 763294 had a related patch set uploaded (by Hashar; author: Hashar):

[mediawiki/core@wmf/1.38.0-wmf.22] Revert \"Optimise Skin::getLanguages()\"

https://gerrit.wikimedia.org/r/763294

I have poked platform engineering team about this issue and I have send the revert patch. I am having lunch and will resume when done.

Revisiting wmf-config/interwiki.php there is:

'_wiki:wikipedia' => '',

That is the sole entry having an empty string as a value. The global entry has:

'__global:wikipedia' => '1 https://en.wikipedia.org/wiki/$1',

The cache is generated by mediawiki/extensions/WikimediaMaintenance script dumpInterwiki.php which might be the culprit. It has:

// Exclude Wikipedia for Wikipedia
$this->makeLink( [ 'iw_prefix' => 'wikipedia', 'iw_url' => null ], "_wiki" );

Revisiting wmf-config/interwiki.php there is:

'_wiki:wikipedia' => '',

That is the sole entry having an empty string as a value. The global entry has:

'__global:wikipedia' => '1 https://en.wikipedia.org/wiki/$1',

The cache is generated by mediawiki/extensions/WikimediaMaintenance script dumpInterwiki.php which might be the culprit.

Those two entries are there for over 5 years (according to git blame), so if it's a problem, it didn't take effect until the code change mentioned above.

I suspect it is a behavior change in ClassicInterwikiLookup::getPregeneratedEntry https://gerrit.wikimedia.org/r/c/mediawiki/core/+/760695/4/includes/interwiki/ClassicInterwikiLookup.php#226 which would not properly take in account the edge case of the url being set to an empty string. Maybe it is an easy fix

I suspect it is a behavior change in ClassicInterwikiLookup::getPregeneratedEntry https://gerrit.wikimedia.org/r/c/mediawiki/core/+/760695/4/includes/interwiki/ClassicInterwikiLookup.php#226 which would not properly take in account the edge case of the url being set to an empty string. Maybe it is an easy fix

Yeah, now it treats an empty string as a valid entry since it is !== false and then does not look up the prefix at global level.

Also the entry with the empty string seems to be intended(?), see https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/WikimediaMaintenance/+/refs/heads/master/dumpInterwiki.php#362.

Change 763503 had a related patch set uploaded (by Zabe; author: Zabe):

[mediawiki/extensions/WikimediaMaintenance@master] Stop excluding 'wikipedia' interwiki prefix

https://gerrit.wikimedia.org/r/763503

Change 763298 had a related patch set uploaded (by Hashar; author: Zabe):

[mediawiki/extensions/WikimediaMaintenance@wmf/1.38.0-wmf.22] Stop excluding the 'wikipedia' interwiki prefix

https://gerrit.wikimedia.org/r/763298

Change 763503 merged by jenkins-bot:

[mediawiki/extensions/WikimediaMaintenance@master] Stop excluding the 'wikipedia' interwiki prefix

https://gerrit.wikimedia.org/r/763503

Change 763488 abandoned by Hashar:

[mediawiki/core@master] Revert \"Optimise Skin::getLanguages()\"

Reason:

Abandoning in favor of fixing our interwiki configuration: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikimediaMaintenance/+/763298

https://gerrit.wikimedia.org/r/763488

Change 763294 abandoned by Hashar:

[mediawiki/core@wmf/1.38.0-wmf.22] Revert \"Optimise Skin::getLanguages()\"

Reason:

Abandoning in favor of fixing our interwiki configuration: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikimediaMaintenance/+/763298

https://gerrit.wikimedia.org/r/763294

Change 763298 merged by jenkins-bot:

[mediawiki/extensions/WikimediaMaintenance@wmf/1.38.0-wmf.22] Stop excluding the 'wikipedia' interwiki prefix

https://gerrit.wikimedia.org/r/763298

Mentioned in SAL (#wikimedia-operations) [2022-02-17T14:32:10Z] <hashar@deploy1002> Synchronized php-1.38.0-wmf.22/extensions/WikimediaMaintenance/dumpInterwiki.php: Backport: [[gerrit:763298|Stop excluding the 'wikipedia' interwiki prefix (T301936)]] (duration: 00m 48s)

Change 763516 had a related patch set uploaded (by Hashar; author: Hashar):

[operations/mediawiki-config@master] Regen interwiki cache to drop erroneous 'wikipedia'

https://gerrit.wikimedia.org/r/763516

Change 763516 merged by jenkins-bot:

[operations/mediawiki-config@master] Regen interwiki cache to drop erroneous 'wikipedia'

https://gerrit.wikimedia.org/r/763516

Patch is on mwdebug1001.

I have purged all the test cases:

Thank you @shizao that has been VERY helpful.

Also purged https://www.mediawiki.org/wiki/User:This,_that_and_the_other/all_interwikis some entries are red links:

mw:abc
mediawikiwiki:abc

I have purged the patch outside of mwdebug1001 and they are still red. So it is a different issue probably cause we are actually on mediawiki.org.

I am deploying the fix

Mentioned in SAL (#wikimedia-operations) [2022-02-17T14:45:23Z] <hashar@deploy1002> Synchronized wmf-config/interwiki.php: Config: [[gerrit:763516|Regen interwiki cache to drop erroneous 'wikipedia' (T301936)]] (duration: 00m 48s)

hashar assigned this task to Zabe.

A huge thank you to @Zabe for the explanation given on https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikimediaMaintenance/+/763503/ and the assistance when I have deployed and regenerated the interwiki cache.

I am very happy we did not have to revert the mediawiki/core patch.

Some page might still show a broken interwiki, they will need a cache purge. It is probably not an issue.

Also purged https://www.mediawiki.org/wiki/User:This,_that_and_the_other/all_interwikis some entries are red links:

mw:abc
mediawikiwiki:abc

I have purged the patch outside of mwdebug1001 and they are still red. So it is a different issue probably cause we are actually on mediawiki.org.

This is the correct behavior. The 'mw' and 'mediawikiwiki' interwikis refer to www.mediawiki.org itself; therefore, these two are really internal links. (This behavior is controlled by $wgLocalInterwikis.) And the page doesn't exist, so they're red.