Page MenuHomePhabricator

LUA getLabelWithLang calls result in many db queries to terms storage with spikey and unpredictable patterns
Closed, ResolvedPublic

Assigned To
Authored By
Addshore
Feb 20 2020, 2:09 PM
Referenced Files
F31629087: image.png
Feb 24 2020, 1:46 PM
F31629090: image.png
Feb 24 2020, 1:46 PM
F31629009: image.png
Feb 24 2020, 12:44 PM
F31629017: image.png
Feb 24 2020, 12:44 PM
F31629019: image.png
Feb 24 2020, 12:44 PM
F31624019: image.png
Feb 20 2020, 2:13 PM
F31624017: image.png
Feb 20 2020, 2:13 PM
F31624021: image.png
Feb 20 2020, 2:13 PM

Description

While monitoring the terms storage situation closely while doing T219123 and T225057 I noticed that there qere some interesting read spikes, went digging and discovered they were directly caused by LUA

Spikes in getLabelWithLang calls:

image.png (840×1 px, 777 KB)

Resulting increases on terms storage related methods:

image.png (615×807 px, 443 KB)

Resulting increases in reads on s8 dbs:

image.png (291×1 px, 249 KB)

We make the assumption that these getLabelWithLang calls are generally not unique and some are more popular than others. For example "the label of countries" used on infoboxes we would expect to be regularly used.

Thus we came to the conclusion that having this call backed by some cache would reduce the termstore lookups.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Change 573565 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/Wikibase@master] Use formatter cache in client LUA label lookups

https://gerrit.wikimedia.org/r/573565

Lots of discussions happened today in mattermost, I have created T245741: Write documentation about the layers of terms caching and decision that were made in the past years which will be used to document the decisions made and the current state for all to see.

The decision for this ticket for now is start with a layer of memcached caching, using the formatter cache.
Once deployed evaluate APC cache and determine if it is desired / needed (already exists for property terms, and actually already merged on master for items but not sure if we will keep it yet)

Change 573565 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Use formatter cache in client LUA label lookups

https://gerrit.wikimedia.org/r/573565

Change 574391 had a related patch set uploaded (by Ladsgroup; owner: Addshore):
[mediawiki/extensions/Wikibase@wmf/1.35.0-wmf.20] Use formatter cache in client LUA label lookups

https://gerrit.wikimedia.org/r/574391

Change 574391 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@wmf/1.35.0-wmf.20] Use formatter cache in client LUA label lookups

https://gerrit.wikimedia.org/r/574391

Mentioned in SAL (#wikimedia-operations) [2020-02-24T12:23:53Z] <ladsgroup@deploy1001> Synchronized php-1.35.0-wmf.20/extensions/Wikibase/client/includes: SWAT: [[gerrit:574391|Use formatter cache in client LUA label lookups (T245740)]] (duration: 00m 56s)

It just absorbed this spike beautifully:

image.png (856×1 px, 171 KB)

image.png (798×907 px, 78 KB)