Reading constraint definitions from the database of another wiki doesn't work
Closed, ResolvedPublic
Actions

Description

here's a wbcheckconstraintsquery on a random item on beta wikidata where I've added a "wrong" value for "sex or gender"

https://wikidata.beta.wmflabs.org/w/api.php?action=wbcheckconstraints&format=json&id=Q424649

as you can see there's a warning

Here's the same thing for a file on beta commons where I've added a "wrong" value for the same thing

https://commons.wikimedia.beta.wmflabs.org/w/api.php?action=wbcheckconstraints&format=json&id=M99899

No warnings. I'd expect warnings. Is there something wrong or is my expectation incorrect?

Details

	Subject	Repo	Branch	Lines +/-
	Make sure constraints are defined for commons as well as wikidata	operations/mediawiki-config	master	+64 -64

Customize query in gerrit

Related Objects

Mentioned In: T244726: WikibaseQualityConstraints can’t cache constraint results in federated setups
Mentioned Here: T245243: Constraints modules not included on pages that aren't in an entity namespace, but that contain entities in a slot
T244726: WikibaseQualityConstraints can’t cache constraint results in federated setups
T240587: Allow reading constraint definitions from the database of other wiki than the local one.

Event Timeline

Cparle created this task.Feb 7 2020, 12:38 PM

Restricted Application added a project: Wikidata. · View Herald TranscriptFeb 7 2020, 12:38 PM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Cparle updated the task description. (Show Details)Feb 7 2020, 12:38 PM

Addshore added a project: Wikidata-Campsite.Feb 7 2020, 12:40 PM

Addshore subscribed.

Hm, it looks to me like the “reading constraint definitions” part itself is working:

lucaswerkmeister-wmde@deployment-deploy01:~$ mwscript shell.php commonswiki
Psy Shell v0.9.12 (PHP 7.2.26-1+0~20191218.33+debian9~1.gbpb5a340+wmf1 — cli) by Justin Hileman
>>> $services = MediaWiki\MediaWikiServices::getInstance()
=> MediaWiki\MediaWikiServices {#196}
>>> $constraintLookup = $services->getService( 'WBQC_ConstraintLookup' )
=> WikibaseQuality\ConstraintReport\CachingConstraintLookup {#2910}
>>> $constraintLookup->queryConstraintsForProperty( new Wikibase\DataModel\Entity\PropertyId( 'P963' ) )
=> [
     WikibaseQuality\ConstraintReport\Constraint {#5356},
     WikibaseQuality\ConstraintReport\Constraint {#5357},
     WikibaseQuality\ConstraintReport\Constraint {#5350},
   ]

Hm, it reports results too:

>>> $dcc = $services->getService( 'WBQC_DelegatingConstraintChecker' )
=> WikibaseQuality\ConstraintReport\ConstraintCheck\DelegatingConstraintChecker {#5346}
>>> $results = $dcc->checkAgainstConstraintsOnEntityId( new Wikibase\MediaInfo\DataModel\MediaInfoId( 'M99899' ) )
=> [
     WikibaseQuality\ConstraintReport\ConstraintCheck\Result\CheckResult {#5616},
     WikibaseQuality\ConstraintReport\ConstraintCheck\Result\CheckResult {#5610},
     WikibaseQuality\ConstraintReport\ConstraintCheck\Result\CheckResult {#5605},
   ]

But it looks like the constraint type is broken?

>>> $results[0]->getConstraint()->getConstraintTypeItemId()
=> "One of"

That’s not an item ID; presumably, that’s why the status is reported as “not implemented”:

>>> $results[0]->getStatus()
=> "todo"

The other two results have real item IDs as constraint types, though, Q505107 and Q505121.

Ah, but Beta Commons has no idea that Q505107 means “one of” – InitialiseSettings-labs.php only configures that for Beta Wikidata:

'wgWBQualityConstraintsOneOfConstraintId' => [
	'wikidatawiki' => 'Q505107',
],

So I guess all the settings that specify entity IDs for WikibaseQualityConstraints (constraint types, severity levels, parameter properties, etc.) need to be changed so that they’re not just configured for Beta Wikidata, but also for Beta Commons.

As it happens, I believe this would not affect real production Commons, because in production we don’t override those settings at all – the default values are those from real Wikidata. So I think constraints might actually work already on Commons if they were enabled there, just not on Beta Commons. (We do override those settings for Test Wikidata; if Test Commons in production is supposed to read constraints from Test Wikidata, then it would need to get those IDs as well.)

In T244572#5859549, @Lucas_Werkmeister_WMDE wrote:
But it looks like the constraint type is broken?
>>> $results[0]->getConstraint()->getConstraintTypeItemId()
=> "One of"
That’s not an item ID; presumably, that’s why the status is reported as “not implemented”:
>>> $results[0]->getStatus()
=> "todo"

This might just be fallout from before we migrated constraint definitions to statements (years ago); it’s possible that we never cleared the constraint definitions table in Beta. If that’s correct, it could be easily fixed by deleting all rows from the constraint definitions table, followed by running the ImportConstraintStatements.php maintenance script.

Hm, it reports results too:
…
the status is reported as “not implemented”

Note that the API won’t return such results by default, but you can ask for them with status=* (see API sandbox for details):

https://commons.wikimedia.beta.wmflabs.org/w/api.php?action=wbcheckconstraints&format=json&id=M99899&status=*

Currently, this returns an internal error, “InvalidArgumentException: $idSerialization must match /^Q[1-9]\\d{0,9}\\z/i”, at CheckResultsRenderer:76:

$typeLabel = $this->entityIdLabelFormatter->formatEntityId( new ItemId( $typeItemId ) );

I assume that’s due to the "One of" constraint type of the first result ("One of" not being a valid Item ID serialization), and it would probably be fixed by clearing and reimporting the constraint definitions table, as mentioned in the previous comment.

Change 570912 had a related patch set uploaded (by Cparle; owner: Cparle):
[operations/mediawiki-config@master] Make sure constraints are defined for commons as well as wikidata

https://gerrit.wikimedia.org/r/570912

gerritbot added a project: Patch-For-Review.Feb 7 2020, 3:52 PM

Addshore moved this task from Incoming to Blocked / Waiting / External on the Wikidata-Campsite board.Feb 8 2020, 7:12 AM

Change 570912 merged by jenkins-bot:
[operations/mediawiki-config@master] Make sure constraints are defined for commons as well as wikidata

https://gerrit.wikimedia.org/r/570912

Hm, now there’s an exception:

InvalidArgumentException: Could not load data from the database of entity source: local

Maintenance_bot removed a project: Patch-For-Review.Feb 10 2020, 11:10 AM

stack trace

InvalidArgumentException at /srv/mediawiki/php-master/extensions/Wikibase/lib/includes/Store/Sql/WikiPageEntityMetaDataLookup.php(237)
#0 /srv/mediawiki/php-master/extensions/Wikibase/lib/includes/Store/Sql/WikiPageEntityMetaDataLookup.php(228): Wikibase\Lib\Store\Sql\WikiPageEntityMetaDataLookup->assertEntityIdFromKnownSource(Wikibase\DataModel\Entity\PropertyId)
#1 /srv/mediawiki/php-master/extensions/Wikibase/lib/includes/Store/Sql/WikiPageEntityMetaDataLookup.php(222): Wikibase\Lib\Store\Sql\WikiPageEntityMetaDataLookup->assertCanHandleEntityId(Wikibase\DataModel\Entity\PropertyId)
#2 /srv/mediawiki/php-master/extensions/Wikibase/lib/includes/Store/Sql/WikiPageEntityMetaDataLookup.php(188): Wikibase\Lib\Store\Sql\WikiPageEntityMetaDataLookup->assertCanHandleEntityIds(array)
#3 /srv/mediawiki/php-master/extensions/Wikibase/lib/includes/Store/Sql/TypeDispatchingWikiPageEntityMetaDataAccessor.php(135): Wikibase\Lib\Store\Sql\WikiPageEntityMetaDataLookup->loadLatestRevisionIds(array, string)
#4 /srv/mediawiki/php-master/extensions/Wikibase/lib/includes/Store/Sql/PrefetchingWikiPageEntityMetaDataAccessor.php(253): Wikibase\Lib\Store\Sql\TypeDispatchingWikiPageEntityMetaDataAccessor->loadLatestRevisionIds(array, string)
#5 /srv/mediawiki/php-master/extensions/WikibaseQualityConstraints/src/Api/CachingResultsSource.php(469): Wikibase\Lib\Store\Sql\PrefetchingWikiPageEntityMetaDataAccessor->loadLatestRevisionIds(array, string)
#6 /srv/mediawiki/php-master/extensions/WikibaseQualityConstraints/src/Api/CachingResultsSource.php(289): WikibaseQuality\ConstraintReport\Api\CachingResultsSource->getLatestRevisionIds(array)
#7 /srv/mediawiki/php-master/extensions/WikibaseQualityConstraints/src/Api/CachingResultsSource.php(239): WikibaseQuality\ConstraintReport\Api\CachingResultsSource->storeResults(Wikibase\MediaInfo\DataModel\MediaInfoId, WikibaseQuality\ConstraintReport\ConstraintCheck\Cache\CachedCheckResults)
#8 /srv/mediawiki/php-master/extensions/WikibaseQualityConstraints/src/Api/CachingResultsSource.php(167): WikibaseQuality\ConstraintReport\Api\CachingResultsSource->getAndStoreResults(array, array, NULL, array)
#9 /srv/mediawiki/php-master/extensions/WikibaseQualityConstraints/src/Api/CheckConstraints.php(172): WikibaseQuality\ConstraintReport\Api\CachingResultsSource->getResults(array, array, NULL, array)
#10 /srv/mediawiki/php-master/includes/api/ApiMain.php(1586): WikibaseQuality\ConstraintReport\Api\CheckConstraints->execute()
#11 /srv/mediawiki/php-master/includes/api/ApiMain.php(522): ApiMain->executeAction()
#12 /srv/mediawiki/php-master/includes/api/ApiMain.php(493): ApiMain->executeActionWithErrorHandling()
#13 /srv/mediawiki/php-master/api.php(84): ApiMain->execute()
#14 /srv/mediawiki/w/api.php(3): require(string)
#15 {main}

CachingResultsSource looks up the latest revision of referenced items and properties so the result can be invalidated when they change. But for that, it gets a service injected that doesn’t support federation, apparently:

ServiceWiring.php

$repo = WikibaseRepo::getDefaultInstance();
$entityIdParser = $repo->getEntityIdParser();
$wikiPageEntityMetaDataAccessor = $repo->getLocalRepoWikiPageMetaDataAccessor();

WikibaseRepo.php

public function getLocalRepoWikiPageMetaDataAccessor() : WikiPageEntityMetaDataAccessor {
	$entityNamespaceLookup = $this->getEntityNamespaceLookup();
	$repoName = ''; // Empty string here means this only works for the local repo
	$dbName = false; // false means the local database

That seems to be the only WikiPageEntityMetaDataAccessor-returning method in WikibaseRepo, though – is there a reason why there’s no federated version?

Anyways – a simple workaround for this would be to disable results caching on Commons, I suppose (WBQualityConstraintsCacheCheckConstraintsResults config).

I created T244726: WikibaseQualityConstraints can’t cache constraint results in federated setups for the bug described above.

I see the cache patch is merged in T244726.
So I figure this should be on the campsite test verification col for now? as the config patch here is also merged?
Thoughts @Lucas_Werkmeister_WMDE

Addshore moved this task from To Do (prioritised from top to bottom) to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Feb 17 2020, 7:54 AM

Maintenance_bot moved this task from incoming to in progress on the Wikidata board.Feb 17 2020, 8:15 AM

Well, there’s not much to verify, we already know the cache change isn’t enough either (T245243). And that other task should itself also be resolved already, I think – the blocker now, at least as far as I’m aware, is that mw.config.get( 'wbIsEditView' ) is undefined on MediaInfo pages and WBQC expects it to be true before doing anything.

Basically, we’re playing whack-a-mole until we’ve discovered all the parts missing to make constraints work on MediaInfo pages, and the Phabricator tasks don’t always fully reflect that. This one has probably outlived its usefulness, at least according to its current title (“reading constraint definitions from the database of another wiki” ought to work now), but I’m not sure if we should close it, retitle it, or something else.

Addshore closed this task as Resolved.Feb 19 2020, 6:59 AM

Addshore claimed this task.

Restricted Application added a project: User-Addshore. · View Herald TranscriptFeb 19 2020, 6:59 AM

Reading constraint definitions from the database of another wiki doesn't workClosed, ResolvedPublicActions

Description

Details

Related Objects

Event Timeline

Reading constraint definitions from the database of another wiki doesn't work
Closed, ResolvedPublic
Actions