Page MenuHomePhabricator

SqlEntityInfoBuilder reads and writes from the old term store regardless of the config
Open, HighPublic8 Story Points

Description

This is a blocker for turning off writing to the old term store and should be fixed ASAP.

SqlEntityInfoBuilder::collectTermsForEntities() that empowers things like property_info table and such, directly reads from the old term store and doesn't respect the config. It queries wb_terms table 10K times per minute on average.

Acceptance Criteria:

  • Should have a cache
  • Should switch read / sql logic based on the existing configuration

Details

Related Gerrit Patches:
mediawiki/extensions/Wikibase : masterWire EntityInfoBuilders
mediawiki/extensions/Wikibase : masterIntroduce ByIdDispatchingEntityInfoBuilder
mediawiki/extensions/Wikibase : masterIntroduce DatabaseEntityInfoBuilder based on the new term store
mediawiki/extensions/Wikibase : wmf/1.35.0-wmf.5Put a layer of APC cache on top of reading wb_terms in SqlEntityInfoBuilder
mediawiki/extensions/Wikibase : masterPut a layer of APC cache on top of reading wb_terms in SqlEntityInfoBuilder

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 28 2019, 12:57 PM
Addshore triaged this task as High priority.Oct 29 2019, 1:54 PM
Addshore updated the task description. (Show Details)
Addshore set the point value for this task to 8.Oct 29 2019, 1:58 PM
Addshore moved this task from Ready to estimate to Ready to pick up on the Wikidata-Campsite board.

Change 546659 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@master] Put a layer of object cache on top of reading wb_terms in SqlEntityInfoBuilder

https://gerrit.wikimedia.org/r/546659

I need to rewrite half of the class to make it work with caching.

Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptMon, Nov 11, 4:23 PM

Change 546659 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Put a layer of APC cache on top of reading wb_terms in SqlEntityInfoBuilder

https://gerrit.wikimedia.org/r/546659

Change 550828 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@wmf/1.35.0-wmf.5] Put a layer of APC cache on top of reading wb_terms in SqlEntityInfoBuilder

https://gerrit.wikimedia.org/r/550828

Change 550828 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@wmf/1.35.0-wmf.5] Put a layer of APC cache on top of reading wb_terms in SqlEntityInfoBuilder

https://gerrit.wikimedia.org/r/550828

Mentioned in SAL (#wikimedia-operations) [2019-11-14T16:04:20Z] <ladsgroup@deploy1001> Synchronized php-1.35.0-wmf.5/extensions/Wikibase: [[gerrit:550828|Put a layer of APC cache on top of reading wb_terms in SqlEntityInfoBuilder]] (T231011 T229407 T236681), Try II (duration: 00m 56s)

The cache is there:

Fixing the cache issue increased the read but not that much:

Change 554057 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@master] Introduce DatabaseEntityInfoBuilder based on the new term store

https://gerrit.wikimedia.org/r/554057

Change 554057 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Introduce DatabaseEntityInfoBuilder based on the new term store

https://gerrit.wikimedia.org/r/554057

Change 554322 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@master] Introduce ByTypeAndIdDispatchingEntityInfoBuilder

https://gerrit.wikimedia.org/r/554322

Change 555523 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@master] [WIP] Wire EntityInfoBuilders

https://gerrit.wikimedia.org/r/555523

Change 554322 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Introduce ByIdDispatchingEntityInfoBuilder

https://gerrit.wikimedia.org/r/554322