Don’t pass overly long entity ID lists to WikiPageEntityDataAccessor
Closed, ResolvedPublic

Description

This was a possible suspected cause of the incident (see parent ticket or incident documentation), but turned out to be wrong. Nevertheless, it’s probably not a good idea to ask the WikiPageEntityDataAccessor for an unlimited number of revisions – let’s limit it to some number. It can be fairly high (perhaps 1000?), just make sure it’s not absolutely ludicrous. If the entity ID set is too large, just disable caching for that entity. (And probably log somewhere.)

Aside – I sure hope we’re deduplicating the entity ID lists somewhere?

Lucas_Werkmeister_WMDE triaged this task as High priority.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 26 2018, 8:29 PM
Lydia_Pintscher moved this task from ready to go to in progress on the Wikidata board.

Change 415054 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/WikibaseQualityConstraints@master] Don’t send huge requests to metadata accessor

https://gerrit.wikimedia.org/r/415054

It can be fairly high (perhaps 1000?), just make sure it’s not absolutely ludicrous.

@Ladsgroup helped me look into this – for Q42, with T188384: Only filter for result statuses after collecting metadata fixed, the dependency metadata spans some 2800 entity IDs (presumably mostly property IDs). So a limit of 1000 would actually be low enough to prevent caching on some entities… the above change uses 10000, which should be fine.

Aside – I sure hope we’re deduplicating the entity ID lists somewhere?

Nope. We’re not. 😨

Change 415054 merged by jenkins-bot:
[mediawiki/extensions/WikibaseQualityConstraints@master] Don’t send huge requests to metadata accessor

https://gerrit.wikimedia.org/r/415054

Change 415582 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/WikibaseQualityConstraints@master] Fix maxRevisionIds handling in CachingResultsBuilder

https://gerrit.wikimedia.org/r/415582

Change 415582 merged by jenkins-bot:
[mediawiki/extensions/WikibaseQualityConstraints@master] Fix maxRevisionIds handling in CachingResultsBuilder

https://gerrit.wikimedia.org/r/415582

Lucas_Werkmeister_WMDE closed this task as Resolved.Mar 2 2018, 11:10 AM
Lucas_Werkmeister_WMDE moved this task from Review to Done on the Wikidata-Sprint-2018-02-28 board.