Page MenuHomePhabricator

Clarify issues around Federation for Machine Vision extension
Open, Needs TriagePublic

Description

As mentioned in T231917, some kind of Wikidata federation is probably necessary for development and testing of this extension. This is because the machine vision API only knows about Wikidata entities, not anything in a local wikibase instance. Communication with Wikidata will likely be required to actually make valid edits to files based on these suggestions, otherwise the properties/values which the user is attempting to add to a file will not correspond to what is available locally.

Event Timeline

egardner created this task.Sep 3 2019, 5:06 PM
Restricted Application added a project: Structured-Data-Backlog. · View Herald TranscriptSep 3 2019, 5:06 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
egardner renamed this task from Clarify issues around Federation to Clarify issues around Federation for Machine Vision extension.Sep 3 2019, 6:05 PM
Cparle added a subscriber: Cparle.Sep 5 2019, 2:13 PM

@Mholloway here's the section of my local config where I turn federation on:

//Federation
//NOTE federation only works if this instance has access to the federated instance's db
//(so you can't federate wikidata from here (and you keep forgetting this, hence the comment))
unset( $wgWBRepoSettings['entityNamespaces']['item'] );
unset( $wgWBRepoSettings['entityNamespaces']['property'] );

$wgWBRepoSettings['useEntitySourceBasedFederation'] = true;
$wgWBClientSettings['useEntitySourceBasedFederation'] = true;
$wgWBClientSettings['sharedCacheKeyPrefix'] = 'local-dev';
$wgWBRepoSettings['sharedCacheKeyPrefix'] = 'local-dev';

$wgWBRepoSettings['foreignRepositories']['wd'] = [
	'repoDatabase' => 'wiki',
	'baseUri' => 'http://127.0.0.1:8080/wiki/Special:EntityData/',
	'supportedEntityTypes' => [ 'item', 'property' ],
	'prefixMapping' => [],
	'entityNamespaces' => [ 'item' => WB_NS_ITEM, 'property' => WB_NS_PROPERTY ]
];
//End Federation

Federation
NOTE federation only works if this instance has access to the federated instance's db
//(so you can't federate wikidata from here (and you keep forgetting this, hence the comment))

Nope, that's clear. That said, it's true I have been writing this code as though we're running in Wikimedia production, and working around the fact that we don't have direct access to the Wikidata DB locally with stuff like https://github.com/wikimedia/mediawiki-extensions-MachineVision/blob/master/src/Handler/LabelResolver.php#L82-L109. I've found this to work well enough for development purposes, which is why I haven't personally been particularly bothered about the federation angle.

Cparle added a comment.Sep 5 2019, 2:30 PM

The "you" in that comment is myself - I tried to federate wikidata from my local env many times, so I needed a comment to remind me why I shouldn't :p

Mholloway added a comment.EditedSep 5 2019, 2:59 PM

Ah, I see :D I thought that seeemed uncharacteristically harsh! Sorry for the misunderstanding.

The other reason I haven't been as strict about writing stuff generically and configurably as I might ordinarily is that setting up a labeling service that works with an arbitrary wikibase is enough work in its own right that it's probably not worth it for a feature with a tight deadline and an uncertain future.