Page MenuHomePhabricator
Create Task
Maniphest T198091

PropertyDataTypeLookupException from line 50 of vendor/wikibase/data-model-services/src/Lookup/EntityRetrievingDataTypeLookup.php: Property data type lookup failed for: P400
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Description

These started appearing in the logs as of 19:07 on 20th June 2018

First apparent occourance: https://logstash.wikimedia.org/app/kibana#/doc/logstash-*/logstash-2018.06.20/mediawiki/?id=AWQemKHcoOODFPKv_v5c

It looks like it is happening at a pretty regular interval https://logstash.wikimedia.org/goto/87b5d23390f454cbeed31066d6b4ce1e

Exception executing job: cirrusSearchLinksUpdate Property:P399 addedLinks=[] cluster=codfw removedLinks=[] requestId=9d506fbd1f4ec71ac82ddd99 : Wikibase\DataModel\Services\Lookup\PropertyDataTypeLookupException: Property data type lookup failed for: P400
&
[9d506fbd1f4ec71ac82ddd99] /rpc/RunSingleJob.php   Wikibase\DataModel\Services\Lookup\PropertyDataTypeLookupException from line 50 of /srv/mediawiki/php-1.32.0-wmf.999/vendor/wikibase/data-model-services/src/Lookup/EntityRetrievingDataTypeLookup.php: Property data type lookup failed for: P400

Stacktrace:

#0 /srv/mediawiki/php-1.32.0-wmf.999/vendor/wikibase/data-model-services/src/Lookup/EntityRetrievingDataTypeLookup.php(37): Wikibase\DataModel\Services\Lookup\EntityRetrievingDataTypeLookup->getProperty(Wikibase\DataModel\Entity\PropertyId)
#1 /srv/mediawiki/php-1.32.0-wmf.999/extensions/Wikibase/lib/includes/PropertyInfoDataTypeLookup.php(53): Wikibase\DataModel\Services\Lookup\EntityRetrievingDataTypeLookup->getDataTypeIdForProperty(Wikibase\DataModel\Entity\PropertyId)
#2 /srv/mediawiki/php-1.32.0-wmf.999/vendor/wikibase/data-model-services/src/Lookup/InProcessCachingDataTypeLookup.php(38): Wikibase\Lib\PropertyInfoDataTypeLookup->getDataTypeIdForProperty(Wikibase\DataModel\Entity\PropertyId)
#3 /srv/mediawiki/php-1.32.0-wmf.999/extensions/Wikibase/repo/includes/Search/Elastic/Fields/StatementsField.php(130): Wikibase\DataModel\Services\Lookup\InProcessCachingDataTypeLookup->getDataTypeIdForProperty(Wikibase\DataModel\Entity\PropertyId)
#4 /srv/mediawiki/php-1.32.0-wmf.999/extensions/Wikibase/repo/includes/Content/EntityHandler.php(763): Wikibase\Repo\Search\Elastic\Fields\StatementsField->getFieldData(Wikibase\DataModel\Entity\Property)
#5 /srv/mediawiki/php-1.32.0-wmf.999/extensions/CirrusSearch/includes/Updater.php(343): Wikibase\Repo\Content\EntityHandler->getDataForSearchIndex(WikiPage, ParserOutput, CirrusSearch)
#6 /srv/mediawiki/php-1.32.0-wmf.999/extensions/CirrusSearch/includes/Updater.php(396): CirrusSearch\Updater::buildDocument(CirrusSearch, WikiPage, CirrusSearch\Connection, integer, integer, integer)
#7 /srv/mediawiki/php-1.32.0-wmf.999/extensions/CirrusSearch/includes/Updater.php(204): CirrusSearch\Updater->buildDocumentsForPages(array, integer)
#8 /srv/mediawiki/php-1.32.0-wmf.999/extensions/CirrusSearch/includes/Updater.php(83): CirrusSearch\Updater->updatePages(array, integer)
#9 /srv/mediawiki/php-1.32.0-wmf.999/extensions/CirrusSearch/includes/Job/LinksUpdate.php(52): CirrusSearch\Updater->updateFromTitle(Title)
#10 /srv/mediawiki/php-1.32.0-wmf.999/extensions/CirrusSearch/includes/Job/Job.php(98): CirrusSearch\Job\LinksUpdate->doJob()
#11 /srv/mediawiki/php-1.32.0-wmf.999/extensions/EventBus/includes/JobExecutor.php(67): CirrusSearch\Job\Job->run()
#12 /srv/mediawiki/rpc/RunSingleJob.php(80): JobExecutor->execute(array)
#13 {main}

Details

SubjectRepoBranchLines +/-
Catch bad lookups and report themmediawiki/extensions/Wikibasemaster+13 -3
Customize query in gerrit

Related Objects

Event Timeline

Addshore created this task.Jun 25 2018, 12:38 PM
Restricted Application added projects: Discovery-ARCHIVED, Discovery-Search. · View Herald TranscriptJun 25 2018, 12:38 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Addshore updated the task description. (Show Details)Jun 25 2018, 12:39 PM
• EBjune added subscribers: Smalyshev, • EBjune.Jun 28 2018, 5:08 PM

@Smalyshev can you take a look at this when you get a chance and see if there's anything we can do on our side? Thanks!

• EBjune triaged this task as Medium priority.Jun 28 2018, 5:11 PM
• EBjune moved this task from needs triage to watching / waiting on the Discovery-Search board.
Smalyshev added a comment.Jun 28 2018, 5:18 PM

Hmm property lookups seem to be old code and P400 is a legit property. Not sure why it would fail. I'll try to dig a bit more into it and maybe add some wrapping so exception does not propagate. In general it seems to make sense that if some of the data is unavailable we could just ignore it for search instead of breaking the whole process.

Smalyshev added a comment.Jun 28 2018, 5:46 PM
This comment was removed by Smalyshev.
Smalyshev added a comment.EditedJun 28 2018, 7:03 PM

Weird thing is that these errors stopped at 2018-06-27T01:07:29 and don't seem to be happening anymore. Also, failed lookup is always for P400 - which is weird since if the lookup has been generally broken, wouldn't lookups for other properties fail at least occasionally? Instead, it's always the job for Property:P399 and always failing for P400. More weirdly, last edit for P399 is June 15, and for P400 is June 9, so I am not sure why these jobs are even touching those in Jun 21-27 timeframe... Looking back, I see other jobs failing but the failure is always for P400. I wonder if there's something magic about it.

Smalyshev added a subscriber: EBernhardson.Jun 28 2018, 7:03 PM
Smalyshev added a comment.Jun 28 2018, 7:12 PM

Digging even more back into the logs, I see stuff like:

[a1fe912ff31262409ada4e58] /rpc/RunSingleJob.php   Wikibase\DataModel\Services\Lookup\PropertyDataTypeLookupException from line 50 of /srv/mediawiki/php-1.32.0-wmf.7/vendor/wikibase/data-model-services/src/Lookup/EntityRetrievingDataTypeLookup.php: Property data type lookup failed for: P74581

P74581 is definitely not a legit property, and I have no idea how we even have something like that in our data. We definitely should put a wrapper on the lookup to filter out such things.

Smalyshev added a comment.Jun 28 2018, 7:32 PM

Just noticed - most of these errors, except for 4, are from testwikidatawiki. That explains weird properties. Since having deleted property on the data is still a legit case, we probably should log it but otherwise ignore it. If it becomes too noisy, we can remove logging and just ignore it.

gerritbot subscribed.Jun 29 2018, 11:23 AM

Change 442912 had a related patch set uploaded (by Addshore; owner: Smalyshev):
[mediawiki/extensions/Wikibase@master] Catch bad lookups and report them

https://gerrit.wikimedia.org/r/442912

gerritbot added a project: Patch-For-Review.Jun 29 2018, 11:23 AM
• Vvjjkkii renamed this task from PropertyDataTypeLookupException from line 50 of vendor/wikibase/data-model-services/src/Lookup/EntityRetrievingDataTypeLookup.php: Property data type lookup failed for: P400 to scaaaaaaaa.Jul 1 2018, 1:03 AM
• Vvjjkkii raised the priority of this task from Medium to High.
• Vvjjkkii added projects: CheckUser, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), Tamil-Sites, Gamepress, Hashtags, Jade, KartoEditor, Language-2018-Apr-June, New-Editor-Experiences, Mail, TCB-Team (now WMDE-TechWish).
• Vvjjkkii updated the task description. (Show Details)
• Vvjjkkii removed subscribers: gerritbot, Aklapper.
CommunityTechBot renamed this task from scaaaaaaaa to PropertyDataTypeLookupException from line 50 of vendor/wikibase/data-model-services/src/Lookup/EntityRetrievingDataTypeLookup.php: Property data type lookup failed for: P400.Jul 2 2018, 10:13 AM
CommunityTechBot lowered the priority of this task from High to Medium.
CommunityTechBot updated the task description. (Show Details)
CommunityTechBot removed projects: TCB-Team (now WMDE-TechWish), Mail, New-Editor-Experiences, Language-2018-Apr-June, KartoEditor, Jade, Hashtags, Gamepress, Tamil-Sites, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), CheckUser.
CommunityTechBot added subscribers: gerritbot, Aklapper.
gerritbot added a comment.Jul 2 2018, 10:42 AM

Change 442912 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Catch bad lookups and report them

https://gerrit.wikimedia.org/r/442912

ReleaseTaggerBot added a project: MW-1.32-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)).Jul 2 2018, 11:00 AM
Smalyshev moved this task from watching / waiting to Current work on the Discovery-Search board.Jul 3 2018, 5:32 PM
Smalyshev edited projects, added Discovery-Search (Current work); removed Discovery-Search.
Smalyshev moved this task from Incoming to Needs Reporting on the Discovery-Search (Current work) board.
debt closed this task as Resolved.Jul 9 2018, 11:39 PM
debt claimed this task.

Moving this to done, but it'll actually go out on the train this week.

• mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:09 PM
Maintenance_bot removed a project: Patch-For-Review.Aug 28 2019, 11:43 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL