InvalidArgumentException: "Could not load data from the database of repository" in invalid wbgetentities calls
Closed, ResolvedPublic

Description

I see a couple of these in the exception logs (with various invalid ids, mostly "Property", but also "Property_talk"):

2016-12-11 19:19:53 [WE2m2QpAAEcAAgMpwBMAAABO] mw1276 wikidatawiki 1.29.0-wmf.5 exception ERROR: [WE2m2QpAAEcAAgMpwBMAAABO] /w/api.php?action=wbgetentities&format=json&ids=PROPERTY:P1201&callback=…   InvalidArgumentException from line 76 of /srv/mediawiki/php-1.29.0-wmf.5/extensions/Wikidata/extensions/Wikibase/lib/includes/Store/Sql/WikiPageEntityMetaDataLookup.php: Could not load data from the database of repository: PROPERTY {"exception_id":"WE2m2QpAAEcAAgMpwBMAAABO"} 
[Exception InvalidArgumentException] (/srv/mediawiki/php-1.29.0-wmf.5/extensions/Wikidata/extensions/Wikibase/lib/includes/Store/Sql/WikiPageEntityMetaDataLookup.php:76) Could not load data from the database of repository: PROPERTY
  #0 /srv/mediawiki/php-1.29.0-wmf.5/extensions/Wikidata/extensions/Wikibase/lib/includes/Store/Sql/PrefetchingWikiPageEntityMetaDataAccessor.php(199): Wikibase\Lib\Store\Sql\WikiPageEntityMetaDataLookup->loadRevisionInformation(array, string)
  #1 /srv/mediawiki/php-1.29.0-wmf.5/extensions/Wikidata/extensions/Wikibase/lib/includes/Store/Sql/PrefetchingWikiPageEntityMetaDataAccessor.php(164): Wikibase\Lib\Store\Sql\PrefetchingWikiPageEntityMetaDataAccessor->doFetch(string)
  #2 /srv/mediawiki/php-1.29.0-wmf.5/extensions/Wikidata/extensions/Wikibase/lib/includes/Store/Sql/WikiPageEntityRevisionLookup.php(84): Wikibase\Lib\Store\Sql\PrefetchingWikiPageEntityMetaDataAccessor->loadRevisionInformation(array, string)
  #3 /srv/mediawiki/php-1.29.0-wmf.5/extensions/Wikidata/extensions/Wikibase/lib/includes/Store/CachingEntityRevisionLookup.php(156): Wikibase\Lib\Store\WikiPageEntityRevisionLookup->getEntityRevision(Wikibase\DataModel\Entity\PropertyId, integer, string)
  #4 /srv/mediawiki/php-1.29.0-wmf.5/extensions/Wikidata/extensions/Wikibase/lib/includes/Store/CachingEntityRevisionLookup.php(138): Wikibase\Lib\Store\CachingEntityRevisionLookup->fetchEntityRevision(Wikibase\DataModel\Entity\PropertyId, integer, string)
  #5 /srv/mediawiki/php-1.29.0-wmf.5/extensions/Wikidata/extensions/Wikibase/lib/includes/Store/CachingEntityRevisionLookup.php(156): Wikibase\Lib\Store\CachingEntityRevisionLookup->getEntityRevision(Wikibase\DataModel\Entity\PropertyId, integer, string)
  #6 /srv/mediawiki/php-1.29.0-wmf.5/extensions/Wikidata/extensions/Wikibase/lib/includes/Store/CachingEntityRevisionLookup.php(138): Wikibase\Lib\Store\CachingEntityRevisionLookup->fetchEntityRevision(Wikibase\DataModel\Entity\PropertyId, integer, string)
  #7 /srv/mediawiki/php-1.29.0-wmf.5/extensions/Wikidata/extensions/Wikibase/repo/includes/Api/GetEntities.php(255): Wikibase\Lib\Store\CachingEntityRevisionLookup->getEntityRevision(Wikibase\DataModel\Entity\PropertyId)
  #8 /srv/mediawiki/php-1.29.0-wmf.5/extensions/Wikidata/extensions/Wikibase/repo/includes/Api/GetEntities.php(237): Wikibase\Repo\Api\GetEntities->getEntityRevision(Wikibase\DataModel\Entity\PropertyId, boolean)
  #9 /srv/mediawiki/php-1.29.0-wmf.5/extensions/Wikidata/extensions/Wikibase/repo/includes/Api/GetEntities.php(129): Wikibase\Repo\Api\GetEntities->getEntityRevisionsFromEntityIds(array, boolean)
  #10 /srv/mediawiki/php-1.29.0-wmf.5/includes/api/ApiMain.php(1434): Wikibase\Repo\Api\GetEntities->execute()
  #11 /srv/mediawiki/php-1.29.0-wmf.5/includes/api/ApiMain.php(510): ApiMain->executeAction()
  #12 /srv/mediawiki/php-1.29.0-wmf.5/includes/api/ApiMain.php(481): ApiMain->executeActionWithErrorHandling()
  #13 /srv/mediawiki/php-1.29.0-wmf.5/api.php(83): ApiMain->execute()
  #14 /srv/mediawiki/w/api.php(3): include(string)
  #15 {main}

Happening quite often:

$ zgrep -cF "Could not load data from the database of repository" archive/exception.log-20161212.gz 
89
hoo created this task.Dec 13 2016, 1:47 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 13 2016, 1:47 PM
WMDE-leszek added a comment.EditedDec 13 2016, 2:23 PM

Silly question, why is it asking for ids=PROPERTY:P120 and not for ids=P120?
While working on federation we decided that using a prefix with a colon would be used to state what repository the entity belongs to. So while asking the API for PROPERTY:P120 lookups tries to access repo PROPERTY which is obviously not configured etc.
Did we miss some important usage here and should not be making such assumption?

thiemowmde triaged this task as Normal priority.Dec 13 2016, 2:41 PM
thiemowmde moved this task from incoming to ready to go on the Wikidata board.
thiemowmde added a project: Easy.
thiemowmde added a subscriber: Lydia_Pintscher.

@WMDE-leszek, this is all perfectly fine. A request with ids=PROPERTY:P120 is supposed to fail. But it should not blow up in the users face with an exception. Instead it should return a useful error message. Something in the line "this is not a valid entity ID".

By the way, I run into the exact same problem on a special page. See https://gerrit.wikimedia.org/r/325777 (already merged) for how it got fixed there.

Restricted Application added a subscriber: TerraCodes. · View Herald TranscriptDec 13 2016, 2:41 PM

Agreed. I'll try to fix the code using the lookup so it handles the exception.

WMDE-leszek added a project: Wikidata-Sprint.

Change 326970 had a related patch set uploaded (by WMDE-leszek):
Make API wbgetentities skip entities from unknown repositories

https://gerrit.wikimedia.org/r/326970

WMDE-leszek moved this task from Proposed to Review on the Wikidata-Sprint board.Dec 13 2016, 3:30 PM

@WMDE-leszek regarding Property:P123: that's the page title of P123. Using page titles instead of entity IDs is a common mistake. In the past, IDs could not contain a oclon, so they would just bounce as malformed. Now they look like foreign IDs from an unknown repo, so they hit a different code path, and fail in a different way. We should indeed make sure that this is handled gracefully.

Btw, I hope nobody gets the idea of using "property" as a repo prefix.... repo prefixes and namespaces should not conflict. But this is really no different from MediaWiki's normal interwiki prefixes. They also compete with wiki namespaces as title prefixes.

thiemowmde moved this task from Review to Doing on the Wikidata-Sprint board.Dec 16 2016, 4:38 PM

Change 328488 had a related patch set uploaded (by WMDE-leszek):
Skip invalid entity ids in PrefetchingWikiPageEntityMetaDataAccessor

https://gerrit.wikimedia.org/r/328488

Change 328488 merged by jenkins-bot:
Skip invalid entity ids in PrefetchingWikiPageEntityMetaDataAccessor

https://gerrit.wikimedia.org/r/328488

Change 326970 merged by jenkins-bot:
Make API wbgetentities skip entities from unknown repositories

https://gerrit.wikimedia.org/r/326970

WMDE-leszek moved this task from Doing to Done on the Wikidata-Sprint board.Dec 29 2016, 11:48 AM
thiemowmde closed this task as Resolved.Jan 3 2017, 1:28 PM