Wikibase uses multiple instances of PrefetchingWikiPageEntityMetaDataAccessor
Closed, ResolvedPublic

Description

As of 6aaba9f1 Wikibase no longer uses a single PrefetchingWikiPageEntityMetaDataAccessor instance, totally breaking the prefetching. This leads to sever slowdowns!

hoo created this task.Feb 6 2017, 9:03 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 6 2017, 9:03 PM

Change 336264 had a related patch set uploaded (by Hoo man):
Revert "Allow optional usage of Client's data retrieval services in the Repo component"

https://gerrit.wikimedia.org/r/336264

If I see right Api\GetEntities is getting the other instance than the entity revision lookups used by WikibaseRepo (via DispatchingEntityRevisionLookup) are using. This is indeed wrong. I am going to try fixing this.

hoo added a comment.Feb 7 2017, 8:04 AM

@WMDE-leszek Thanks. The dumpers are also very important here… if you can't find a viable solution for backport, please consider my revert.

And dumpers yes, I didn't think they might be an issue too but they're probably even more important in this regard.

Change 336387 had a related patch set uploaded (by WMDE-leszek):
Provide EntityPrefetcher in EntityDataRetrievalServiceFactory

https://gerrit.wikimedia.org/r/336387

WMDE-leszek moved this task from Proposed to Review on the Wikidata-Sprint board.Feb 7 2017, 9:54 AM

Change 336387 merged by jenkins-bot:
Provide EntityPrefetcher in EntityDataRetrievalServiceFactory

https://gerrit.wikimedia.org/r/336387

hoo closed this task as "Resolved".Feb 7 2017, 6:16 PM
hoo moved this task from Review to Done on the Wikidata-Sprint board.
hoo removed a project: Patch-For-Review.

Manually verified that the dumpers use proper prefetching again.

We probably want to backport this.

Change 336452 had a related patch set uploaded (by Daniel Kinzler):
Revert "Allow optional usage of Client's data retrieval services in the Repo component"

https://gerrit.wikimedia.org/r/336452

Change 336452 merged by jenkins-bot:
Revert "Allow optional usage of Client's data retrieval services in the Repo component"

https://gerrit.wikimedia.org/r/336452

Change 336264 abandoned by Hoo man:
Revert "Allow optional usage of Client's data retrieval services in the Repo component"

Reason:
No longer relevant.

https://gerrit.wikimedia.org/r/336264

Will get backported today at 15:00 UTC (16:00 CET).

Mentioned in SAL (#wikimedia-operations) [2017-02-08T15:15:26Z] <hoo@tin> Synchronized php-1.29.0-wmf.11/extensions/Wikidata: Wikibase uses multiple EntityPrefetchers (T157380) (duration: 02m 11s)

hoo added a comment.Feb 8 2017, 3:17 PM

Time to dump testwikidatawiki went from 1m21.433s to 0m34.786s with the prefetcher fix.

Mentioned in SAL (#wikimedia-operations) [2017-02-08T15:20:56Z] <hoo@tin> Synchronized php-1.29.0-wmf.10/extensions/Wikidata: Wikibase uses multiple EntityPrefetchers (T157380) (duration: 02m 07s)