⚓ T242096 SingleEntitySourceServices PrefetchingTermLookup should use new terms storage

Subject	Repo	Branch	Lines +/-
wbterms: Update SingleEntitySourceServices PrefetchingTermLookup	mediawiki/extensions/Wikibase	wmf/1.35.0-wmf.14	+140 -75
Use new namespace for PrefetchingTermLookup	mediawiki/extensions/WikibaseMediaInfo	master	+1 -1
wbterms: Update SingleEntitySourceServices PrefetchingTermLookup	mediawiki/extensions/Wikibase	master	+139 -76
Update SingleEntitySourceServices to respect item terms migration stages	mediawiki/extensions/Wikibase	master	+117 -16
wb_terms, delete unused MultiTermStoreWriter	mediawiki/extensions/Wikibase	master	+0 -92

Status	Assigned	Task
Resolved	Addshore	T208425 [EPIC] Kill the wb_terms table
Resolved	Addshore	T219303 [Checkpoint 5] Update Read Logic
Duplicate	None	T236979 Fix remaining usecases still reading from old wb_terms schema
Resolved	Addshore	T232040 Add label and description collision detectors for new terms store
Resolved	Addshore	T242096 SingleEntitySourceServices PrefetchingTermLookup should use new terms storage

Addshore created this task.Jan 7 2020, 11:18 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 7 2020, 11:18 AM

Addshore moved this task from Incoming to Ready to pick up on the Wikidata-Campsite board.Jan 7 2020, 11:18 AM

Addshore mentioned this in T208425: [EPIC] Kill the wb_terms table.

Addshore added a parent task: T232040: Add label and description collision detectors for new terms store.Jan 7 2020, 11:38 AM

Change 562320 had a related patch set uploaded (by Addshore; owner: Sarhan):
[mediawiki/extensions/Wikibase@master] Update SingleEntitySourceServices to respect item terms migration stages

https://gerrit.wikimedia.org/r/562320

gerritbot added a project: Patch-For-Review.Jan 7 2020, 11:38 AM

Addshore claimed this task.Jan 7 2020, 11:48 AM

Restricted Application added a project: User-Addshore. · View Herald TranscriptJan 7 2020, 11:48 AM

I don't want to mess with the WIP patch but I want to mention that the patch at its current state removes usage of BufferingTermLookup, this caching layer is widely needed and if you look at the numbers in grafana, 62% of requests only hit this caching layer (it used to be higher IIRC). Given that the new term store will have slower reads (which is by design) I think we should keep this caching layer.

Addshore moved this task from Ready to pick up to Wikidata-Campsite-Iteration-∞ (On Hold) on the Wikidata-Campsite board.Jan 7 2020, 1:30 PM

Addshore edited projects, added Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)); removed Wikidata-Campsite.

Addshore moved this task from To Do (prioritised from top to bottom) to Doing on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Jan 7 2020, 1:32 PM

Maintenance_bot moved this task from incoming to in progress on the Wikidata board.Jan 7 2020, 2:15 PM

Change 562548 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/Wikibase@master] wb_terms, delete unused MultiTermStoreWriter

https://gerrit.wikimedia.org/r/562548

Change 562548 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] wb_terms, delete unused MultiTermStoreWriter

https://gerrit.wikimedia.org/r/562548

In T242096#5781667, @Ladsgroup wrote:

I don't want to mess with the WIP patch but I want to mention that the patch at its current state removes usage of BufferingTermLookup, this caching layer is widely needed and if you look at the numbers in grafana, 62% of requests only hit this caching layer (it used to be higher IIRC). Given that the new term store will have slower reads (which is by design) I think we should keep this caching layer.

Never mind, apparently PrefetchingPropertyTermLookup has the caching layer as $terms attribute. I think it's fine (unless it doesn't work as intended which I hope not.)

ReleaseTaggerBot added a project: MW-1.35-notes (1.35.0-wmf.14; 2020-01-07).Jan 7 2020, 6:00 PM

Ladsgroup mentioned this in T241975: entitysources: Remove old MultiRepository & PerRepository Service containers and config.Jan 7 2020, 6:57 PM

Addshore mentioned this in T242326: Wikibase Client PageTerms api module should not use SqlTermIndex directly.Jan 9 2020, 9:31 AM

Addshore moved this task from Unsorted 💣 to Active 🚁 on the User-Addshore board.Jan 9 2020, 9:49 AM

Change 563204 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/Wikibase@master] Update SingleEntitySourceServices to respect item terms migration stages

https://gerrit.wikimedia.org/r/563204

Change 563204 abandoned by Addshore:
Update SingleEntitySourceServices to respect item terms migration stages

https://gerrit.wikimedia.org/r/563204

Change 563227 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/WikibaseMediaInfo@master] Use new namespace for PrefetchingTermLookup

https://gerrit.wikimedia.org/r/563227

Addshore moved this task from Doing to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Jan 9 2020, 7:48 PM

Change 562320 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] wbterms: Update SingleEntitySourceServices PrefetchingTermLookup

https://gerrit.wikimedia.org/r/562320

ReleaseTaggerBot edited projects, added MW-1.35-notes (1.35.0-wmf.15; 2020-01-14); removed MW-1.35-notes (1.35.0-wmf.14; 2020-01-07).Jan 10 2020, 8:00 AM

Change 563227 merged by jenkins-bot:
[mediawiki/extensions/WikibaseMediaInfo@master] Use new namespace for PrefetchingTermLookup

https://gerrit.wikimedia.org/r/563227

Maintenance_bot removed a project: Patch-For-Review.Jan 10 2020, 10:10 AM

Ladsgroup moved this task from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Jan 10 2020, 10:17 AM

Change 563958 had a related patch set uploaded (by Ladsgroup; owner: Addshore):
[mediawiki/extensions/Wikibase@wmf/1.35.0-wmf.14] wbterms: Update SingleEntitySourceServices PrefetchingTermLookup

https://gerrit.wikimedia.org/r/563958

gerritbot added a project: Patch-For-Review.Jan 13 2020, 8:39 AM

Addshore triaged this task as Medium priority.Jan 14 2020, 9:54 AM

Change 563958 abandoned by Addshore:
wbterms: Update SingleEntitySourceServices PrefetchingTermLookup

Reason:
Abandoning as I figure we are not backporting this!

https://gerrit.wikimedia.org/r/563958

Maintenance_bot removed a project: Patch-For-Review.Jan 14 2020, 12:10 PM

Addshore closed this task as Resolved.Jan 15 2020, 5:08 PM