Page MenuHomePhabricator

PropertyDataTypeLookupException from line 50 of vendor/wikibase/data-model-services/src/Lookup/EntityRetrievingDataTypeLookup.php: Property data type lookup failed for: P400
Closed, ResolvedPublicPRODUCTION ERROR

Description

These started appearing in the logs as of 19:07 on 20th June 2018

First apparent occourance: https://logstash.wikimedia.org/app/kibana#/doc/logstash-*/logstash-2018.06.20/mediawiki/?id=AWQemKHcoOODFPKv_v5c

It looks like it is happening at a pretty regular interval https://logstash.wikimedia.org/goto/87b5d23390f454cbeed31066d6b4ce1e

Exception executing job: cirrusSearchLinksUpdate Property:P399 addedLinks=[] cluster=codfw removedLinks=[] requestId=9d506fbd1f4ec71ac82ddd99 : Wikibase\DataModel\Services\Lookup\PropertyDataTypeLookupException: Property data type lookup failed for: P400
&
[9d506fbd1f4ec71ac82ddd99] /rpc/RunSingleJob.php   Wikibase\DataModel\Services\Lookup\PropertyDataTypeLookupException from line 50 of /srv/mediawiki/php-1.32.0-wmf.999/vendor/wikibase/data-model-services/src/Lookup/EntityRetrievingDataTypeLookup.php: Property data type lookup failed for: P400

Stacktrace:

#0 /srv/mediawiki/php-1.32.0-wmf.999/vendor/wikibase/data-model-services/src/Lookup/EntityRetrievingDataTypeLookup.php(37): Wikibase\DataModel\Services\Lookup\EntityRetrievingDataTypeLookup->getProperty(Wikibase\DataModel\Entity\PropertyId)
#1 /srv/mediawiki/php-1.32.0-wmf.999/extensions/Wikibase/lib/includes/PropertyInfoDataTypeLookup.php(53): Wikibase\DataModel\Services\Lookup\EntityRetrievingDataTypeLookup->getDataTypeIdForProperty(Wikibase\DataModel\Entity\PropertyId)
#2 /srv/mediawiki/php-1.32.0-wmf.999/vendor/wikibase/data-model-services/src/Lookup/InProcessCachingDataTypeLookup.php(38): Wikibase\Lib\PropertyInfoDataTypeLookup->getDataTypeIdForProperty(Wikibase\DataModel\Entity\PropertyId)
#3 /srv/mediawiki/php-1.32.0-wmf.999/extensions/Wikibase/repo/includes/Search/Elastic/Fields/StatementsField.php(130): Wikibase\DataModel\Services\Lookup\InProcessCachingDataTypeLookup->getDataTypeIdForProperty(Wikibase\DataModel\Entity\PropertyId)
#4 /srv/mediawiki/php-1.32.0-wmf.999/extensions/Wikibase/repo/includes/Content/EntityHandler.php(763): Wikibase\Repo\Search\Elastic\Fields\StatementsField->getFieldData(Wikibase\DataModel\Entity\Property)
#5 /srv/mediawiki/php-1.32.0-wmf.999/extensions/CirrusSearch/includes/Updater.php(343): Wikibase\Repo\Content\EntityHandler->getDataForSearchIndex(WikiPage, ParserOutput, CirrusSearch)
#6 /srv/mediawiki/php-1.32.0-wmf.999/extensions/CirrusSearch/includes/Updater.php(396): CirrusSearch\Updater::buildDocument(CirrusSearch, WikiPage, CirrusSearch\Connection, integer, integer, integer)
#7 /srv/mediawiki/php-1.32.0-wmf.999/extensions/CirrusSearch/includes/Updater.php(204): CirrusSearch\Updater->buildDocumentsForPages(array, integer)
#8 /srv/mediawiki/php-1.32.0-wmf.999/extensions/CirrusSearch/includes/Updater.php(83): CirrusSearch\Updater->updatePages(array, integer)
#9 /srv/mediawiki/php-1.32.0-wmf.999/extensions/CirrusSearch/includes/Job/LinksUpdate.php(52): CirrusSearch\Updater->updateFromTitle(Title)
#10 /srv/mediawiki/php-1.32.0-wmf.999/extensions/CirrusSearch/includes/Job/Job.php(98): CirrusSearch\Job\LinksUpdate->doJob()
#11 /srv/mediawiki/php-1.32.0-wmf.999/extensions/EventBus/includes/JobExecutor.php(67): CirrusSearch\Job\Job->run()
#12 /srv/mediawiki/rpc/RunSingleJob.php(80): JobExecutor->execute(array)
#13 {main}

Event Timeline

Restricted Application added projects: Discovery, Discovery-Search. · View Herald TranscriptJun 25 2018, 12:38 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Addshore updated the task description. (Show Details)Jun 25 2018, 12:39 PM

@Smalyshev can you take a look at this when you get a chance and see if there's anything we can do on our side? Thanks!

EBjune triaged this task as Medium priority.Jun 28 2018, 5:11 PM
EBjune moved this task from needs triage to watching / waiting on the Discovery-Search board.

Hmm property lookups seem to be old code and P400 is a legit property. Not sure why it would fail. I'll try to dig a bit more into it and maybe add some wrapping so exception does not propagate. In general it seems to make sense that if some of the data is unavailable we could just ignore it for search instead of breaking the whole process.

This comment was removed by Smalyshev.
Smalyshev added a comment.EditedJun 28 2018, 7:03 PM

Weird thing is that these errors stopped at 2018-06-27T01:07:29 and don't seem to be happening anymore. Also, failed lookup is always for P400 - which is weird since if the lookup has been generally broken, wouldn't lookups for other properties fail at least occasionally? Instead, it's always the job for Property:P399 and always failing for P400. More weirdly, last edit for P399 is June 15, and for P400 is June 9, so I am not sure why these jobs are even touching those in Jun 21-27 timeframe... Looking back, I see other jobs failing but the failure is always for P400. I wonder if there's something magic about it.

Digging even more back into the logs, I see stuff like:

[a1fe912ff31262409ada4e58] /rpc/RunSingleJob.php   Wikibase\DataModel\Services\Lookup\PropertyDataTypeLookupException from line 50 of /srv/mediawiki/php-1.32.0-wmf.7/vendor/wikibase/data-model-services/src/Lookup/EntityRetrievingDataTypeLookup.php: Property data type lookup failed for: P74581

P74581 is definitely not a legit property, and I have no idea how we even have something like that in our data. We definitely should put a wrapper on the lookup to filter out such things.

Just noticed - most of these errors, except for 4, are from testwikidatawiki. That explains weird properties. Since having deleted property on the data is still a legit case, we probably should log it but otherwise ignore it. If it becomes too noisy, we can remove logging and just ignore it.

Change 442912 had a related patch set uploaded (by Addshore; owner: Smalyshev):
[mediawiki/extensions/Wikibase@master] Catch bad lookups and report them

https://gerrit.wikimedia.org/r/442912

Vvjjkkii renamed this task from PropertyDataTypeLookupException from line 50 of vendor/wikibase/data-model-services/src/Lookup/EntityRetrievingDataTypeLookup.php: Property data type lookup failed for: P400 to scaaaaaaaa.Jul 1 2018, 1:03 AM
Vvjjkkii raised the priority of this task from Medium to High.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed subscribers: gerritbot, Aklapper.
CommunityTechBot renamed this task from scaaaaaaaa to PropertyDataTypeLookupException from line 50 of vendor/wikibase/data-model-services/src/Lookup/EntityRetrievingDataTypeLookup.php: Property data type lookup failed for: P400.Jul 2 2018, 10:13 AM
CommunityTechBot lowered the priority of this task from High to Medium.
CommunityTechBot updated the task description. (Show Details)
CommunityTechBot added subscribers: gerritbot, Aklapper.

Change 442912 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Catch bad lookups and report them

https://gerrit.wikimedia.org/r/442912

debt closed this task as Resolved.Jul 9 2018, 11:39 PM
debt claimed this task.

Moving this to done, but it'll actually go out on the train this week.

mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:09 PM