Page MenuHomePhabricator

LUA getLabelWithLang calls result in many db queries to terms storage with spikey and unpredictable patterns
Closed, ResolvedPublic

Description

While monitoring the terms storage situation closely while doing T219123 and T225057 I noticed that there qere some interesting read spikes, went digging and discovered they were directly caused by LUA

Spikes in getLabelWithLang calls:

Resulting increases on terms storage related methods:

Resulting increases in reads on s8 dbs:

We make the assumption that these getLabelWithLang calls are generally not unique and some are more popular than others. For example "the label of countries" used on infoboxes we would expect to be regularly used.

Thus we came to the conclusion that having this call backed by some cache would reduce the termstore lookups.

Event Timeline

Addshore created this task.Feb 20 2020, 2:09 PM
Restricted Application added a project: User-Addshore. · View Herald TranscriptFeb 20 2020, 2:09 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Change 573565 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/Wikibase@master] Use formatter cache in client LUA label lookups

https://gerrit.wikimedia.org/r/573565

Restricted Application added a project: Wikidata. · View Herald TranscriptFeb 20 2020, 2:13 PM

Lots of discussions happened today in mattermost, I have created T245741: Write ADR and documentation about the layers of terms caching and decision that were made in the past years which will be used to document the decisions made and the current state for all to see.

The decision for this ticket for now is start with a layer of memcached caching, using the formatter cache.
Once deployed evaluate APC cache and determine if it is desired / needed (already exists for property terms, and actually already merged on master for items but not sure if we will keep it yet)

Tarrow updated the task description. (Show Details)Feb 20 2020, 2:29 PM

Change 573565 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Use formatter cache in client LUA label lookups

https://gerrit.wikimedia.org/r/573565

Change 574391 had a related patch set uploaded (by Ladsgroup; owner: Addshore):
[mediawiki/extensions/Wikibase@wmf/1.35.0-wmf.20] Use formatter cache in client LUA label lookups

https://gerrit.wikimedia.org/r/574391

Change 574391 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@wmf/1.35.0-wmf.20] Use formatter cache in client LUA label lookups

https://gerrit.wikimedia.org/r/574391

Mentioned in SAL (#wikimedia-operations) [2020-02-24T12:23:53Z] <ladsgroup@deploy1001> Synchronized php-1.35.0-wmf.20/extensions/Wikibase/client/includes: SWAT: [[gerrit:574391|Use formatter cache in client LUA label lookups (T245740)]] (duration: 00m 56s)

It just absorbed this spike beautifully:


Addshore closed this task as Resolved.Feb 25 2020, 10:31 PM