Per T205045#4937780 we should alter the config var in production to be an even smaller value.
The wikibase config options is "addEntityUsagesBatchSize"
Docs can be seen @ https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Wikibase/+/485849/5/docs/options.wiki
Default value is currently 300
We should try decreasing this value to 100, and monitor the occurrence of errors.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T30599 Deadlock tracking bug (tracking) | |||
| Resolved | PRODUCTION ERROR | Ladsgroup | T205045 Exception from LinksUpdate: Deadlock found in database query (from Wikibase\Client\Usage\Sql\EntityUsageTable::addUsages) | ||
| Resolved | Michael | T225500 Decrease EntityUsageTable addUsage batch size to 100 |
Change 516478 had a related patch set uploaded (by Michael Große; owner: Michael Große):
[operations/mediawiki-config@master] Set EntityUsageTable addUsage batch size to 100
The plan is to remove it to 100 in steps of 200, 150, 100.
While doing that, we have to watch https://grafana.wikimedia.org/d/000000273/mysql?orgId=1 especially the "Queries per second" number. That number is 13.77K right now.
A DBA should be aware of these changes.
Change 516478 merged by jenkins-bot:
[operations/mediawiki-config@master] Set EntityUsageTable addUsage batch size to 200
Mentioned in SAL (#wikimedia-operations) [2019-06-17T11:49:21Z] <awight@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit|516478 Set EntityUsageTable addUsage batch size to 200 (T225500)]] (duration: 00m 47s)
Change 517669 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/mediawiki-config@master] Set EntityUsageTable addUsage batch size to 150
Change 517669 merged by jenkins-bot:
[operations/mediawiki-config@master] Set EntityUsageTable addUsage batch size to 150
Mentioned in SAL (#wikimedia-operations) [2019-06-20T14:22:48Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:517669|Set EntityUsageTable addUsage batch size to 150 (T225500)]] (duration: 00m 56s)
Change 518952 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/mediawiki-config@master] Set EntityUsageTable addUsage batch size to 100
Change 518952 merged by jenkins-bot:
[operations/mediawiki-config@master] Set EntityUsageTable addUsage batch size to 100
Mentioned in SAL (#wikimedia-operations) [2019-06-26T12:06:36Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:518952|Set EntityUsageTable addUsage batch size to 100 (T225500)]] (duration: 00m 56s)
https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&from=now-7d&to=now
No noticeable increase in QPS in the last 6 days https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&from=now-7d&to=now (3 days before deploying this change until today).