Beta should not use productions interwiki.php
Closed, ResolvedPublic

Description

Beta should overwrite $wgInterwikiCache to point to a labs specific cdb.

That might break some stuff on beta relying on the current behavior, but that would also make beta more production like (eg. transwiki-imports from a beta wiki to another will work then).


Version: unspecified
Severity: normal

Details

Reference
bz67931
bzimport raised the priority of this task from to Normal.
bzimport set Reference to bz67931.
bzimport added a subscriber: Unknown Object (MLST).
hoo created this task.Jul 12 2014, 7:41 PM
hoo added a comment.Jul 16 2014, 2:19 PM

Alternatively we could provide extra interwiki prefixes for labs sites (if those don't exist already)... like beta-de (or so). That would also allow importing from labs.

greg added a comment.Jul 16 2014, 5:03 PM

Antoine: would this break any of the auto-fancy stuff like where we fetch an image from prod commons if we need it?

I have absolute no idea how interwikis are generated nor how they are cached. Maybe the interwiki.cdb file generated on beta uses the list of beta wikis and is thus up to date?

(In reply to Greg Grossmeier from comment #2)

Antoine: would this break any of the auto-fancy stuff like where we fetch an
image from prod commons if we need it?

That is done by setting $wgUseInstantCommons = true which has commons URL hardcoded directly in includes/Setup.php

if ( $wgUseInstantCommons ) {

$wgForeignFileRepos[] = array(
    'class' => 'ForeignAPIRepo',
    'name' => 'wikimediacommons',
    'apibase' => WebRequest::detectProtocol() === 'https' ?
        'https://commons.wikimedia.org/w/api.php' :
        'http://commons.wikimedia.org/w/api.php',
    ...
);

}

So the auto-fancy fetching of missing images will still work :]

hoo added a comment.Jul 22 2014, 8:52 AM

(In reply to Antoine "hashar" Musso from comment #3)

I have absolute no idea how interwikis are generated nor how they are
cached. Maybe the interwiki.cdb file generated on beta uses the list of
beta wikis and is thus up to date?

No, it's the interwiki.cdb from production (it's taken out of the mediawiki-configuration repo)

greg assigned this task to Reedy.Nov 25 2014, 12:07 AM
greg moved this task from To Triage to Next: Maintenance on the Beta-Cluster-Infrastructure board.
Reedy added a comment.Nov 25 2014, 5:26 PM

It's probably easiest to just run the script on beta, create it as interwiki-labs.cdb, commit it...

if ( file_exists( "$wmfConfigDir/interwiki.cdb" ) ) {
	$wgInterwikiCache = "$wmfConfigDir/interwiki.cdb";
}

to

$iwCacheFile = getRealmSpecificFilename( "$wmfConfigDir/interwiki.cdb" );
if ( file_exists( $iwCacheFile ) ) {
	$wgInterwikiCache = $iwCacheFile;
}
unset( $iwCacheFile );

Should be trivial

Reedy added a comment.EditedNov 25 2014, 5:37 PM

https://gerrit.wikimedia.org/r/175755

The question is whether dumpInterwiki.php makes a file that's actually useful for labs... I'll have to have a look at what config it actually makes

faidon removed a subscriber: faidon.Dec 3 2014, 1:12 PM
jayvdb added a subscriber: jayvdb.Jan 16 2015, 9:37 AM
Reedy added a comment.Feb 3 2015, 5:28 PM

We shouldn't be committing this to mediawiki-config, see also T75905

hashar removed a subscriber: hashar.Feb 4 2015, 9:00 AM
hashar removed Reedy as the assignee of this task.
hashar set Security to None.
hashar added a subscriber: hashar.

@Reedy is not working on this task.

hashar moved this task from In-progress to To Triage on the Release-Engineering-Team board.

Change 175755 had a related patch set uploaded (by Greg Grossmeier):
Add interwiki-labs.cdb

https://gerrit.wikimedia.org/r/175755

Change 175755 merged by jenkins-bot:
Add interwiki-labs.cdb

https://gerrit.wikimedia.org/r/175755

Krenair closed this task as Resolved.Jul 15 2015, 11:25 PM
Krenair claimed this task.

Sorry for the silly question, but ... if that change has been deployed to the beta cluster, why hasnt the sitematrix and interwikimap updated?
Is there something else to be done, or is this a 'wait a while and it will fix itself' problem?
I note that https://noc.wikimedia.org/conf/interwiki-labs.cdb is currently a 403 Forbidden.

Sorry for the silly question, but ... if that change has been deployed to the beta cluster, why hasnt the sitematrix and interwikimap updated?

That change doesn't actually create the interwiki-labs.cdb in beta... Probably needs a labs-equivalent version of updateinterwikicache to be run on deployment-bastion?

I note that https://noc.wikimedia.org/conf/interwiki-labs.cdb is currently a 403 Forbidden.

Yeah, I doubt that's going to work as noc.wikimedia.org can only access the production mediawiki files. If we had a noc.wikimedia.beta.wmflabs.org it would only be able to show interwiki-labs.cdb rather than interwiki.cdb.

Reedy added a comment.Jul 16 2015, 3:32 AM

An interwiki.cdb for beta can't be generated (and expected to work correctly) until T75906 is fixed

jayvdb reopened this task as Open.Jul 16 2015, 3:37 AM

Re-opening as this has incomplete sub-tasks, and has tasks blocked on this being fixed.

Fwiw, the tasks depending on this task do not depend on the issues raised in T75906#1011937 , as Pywikibot only chokes on the fact that the beta interwikimap link the language prefixes to http://aa.wikipedia.org/ instead of http://aa.wikipedia.beta.wmflabs.org/ .

The JSON currently looks like this on beta:

{
    "query": {
        "interwikimap": [
            {
                "prefix": "aa",
                "local": "",
                "language": "Qaf\u00e1r af",
                "url": "http://aa.wikipedia.org/wiki/$1",
                "protorel": ""
            },
}

Would running updateinterwikicache now fix that?

We did run updateinterwikicache, and now there is an interwiki map (rather than nothing, and coming from interwiki-labs.cdb), but it has links to production sites, which I believe would be part of T75906.

Mattflaschen-WMF reopened this task as Open.Aug 31 2017, 2:08 AM
Mattflaschen-WMF claimed this task.

This was reverted (and thus re-opened) on Sep 14, 2015: rOMWC0feb37f45a89: Revert "Add interwiki-labs.cdb".

Mattflaschen-WMF renamed this task from Beta should not use productions interwiki.cdb to Beta should not use productions interwiki.php.Aug 31 2017, 2:25 AM

Change 374922 had a related patch set uploaded (by Mattflaschen; owner: Mattflaschen):
[operations/mediawiki-config@master] Fix interwiki links on Labs

https://gerrit.wikimedia.org/r/374922

Now that interwiki.php is committed and the script works fine, this should be fine.

Change 374922 merged by jenkins-bot:
[operations/mediawiki-config@master] Fix interwiki links on Beta Cluster

https://gerrit.wikimedia.org/r/374922

Mentioned in SAL (#wikimedia-operations) [2017-08-31T18:51:51Z] <mattflaschen@tin> Synchronized wmf-config: Interwiki links on Beta Cluster: T69931 (duration: 00m 49s)

Mentioned in SAL (#wikimedia-operations) [2017-08-31T18:53:11Z] <mattflaschen@tin> Synchronized docroot/noc/conf/interwiki-labs.php.txt: Interwiki links on Beta Cluster: T69931 (duration: 00m 47s)

Mattflaschen-WMF closed this task as Resolved.Aug 31 2017, 10:39 PM