Page MenuHomePhabricator

Exception from LinksUpdate "Could not acquire lock for page" when a page is edited frequently
Closed, ResolvedPublicPRODUCTION ERROR

Description

Error

Request ID: W7bPjArAEHYAAC2B6EYAAABG

message
RuntimeException: Could not acquire lock for page ID '6548940'.
trace
#0 /srv/mediawiki/php-1.32.0-wmf.24/includes/deferred/DeferredUpdates.php(268): LinksUpdate->doUpdate()
#1 /srv/mediawiki/php-1.32.0-wmf.24/includes/deferred/DeferredUpdates.php(214): DeferredUpdates::runUpdate(LinksUpdate, Wikimedia\Rdbms\LBFactoryMulti, string, integer)
#2 /srv/mediawiki/php-1.32.0-wmf.24/includes/deferred/DeferredUpdates.php(134): DeferredUpdates::execute(array, string, integer)
#3 /srv/mediawiki/php-1.32.0-wmf.24/includes/MediaWiki.php(914): DeferredUpdates::doUpdates(string)
#4 /srv/mediawiki/php-1.32.0-wmf.24/includes/MediaWiki.php(734): MediaWiki->restInPeace(string, boolean)
#5 [internal function]: Closure$MediaWiki::doPostOutputShutdown()

Impact

When the same page is edited repeatedly in a short period of time, the LinksUpdate process can fail.

This particular exception was from edits at https://en.wikipedia.org/w/index.php?title=Wikipedia:Version_1.0_Editorial_Team/Biography_(military)_articles_by_quality_log&offset=20181005024213&action=history. Due to the high edit frequency there, the locks were unavailable and some of the edits had their update fail. When this happens for the "last" edit, which is not unlikely, it means the database will remain indefinitely in an incorrect state.

Derived data such as used by category membership, language links, WhatLinksHere, and search, may return outdated or incorrect information about the page in question, and does not automatically recover in an eventual way.

Notes

A similar issue was reported from RefreshLink jobs on the JobQueue as T170596, which we fixed. However, the issue still happens when multiple edits are made to the same page by the same or different users.

Event Timeline

Krinkle moved this task from Doing (old) to Backlog: Maintenance on the Performance-Team board.

Logstash query +message:"Could not acquire lock for" on mediawiki-errors shows a few reports still in last 7 days (mainly enwiki and commons, API requests and jobs). Focussing on T206283 for now first, and leaving this up for grabs for someone else, or myself at a later time :)

Krinkle triaged this task as Medium priority.Dec 19 2018, 11:43 PM

Change 466980 had a related patch set uploaded (by Daniel Kinzler; owner: Aaron Schulz):
[mediawiki/core@master] Split out new RefreshSecondaryDataUpdate class

https://gerrit.wikimedia.org/r/466980

Change 466981 had a related patch set uploaded (by Daniel Kinzler; owner: Aaron Schulz):
[mediawiki/core@master] Make DeferredUpdates enqueue updates that failed to run when possible

https://gerrit.wikimedia.org/r/466981

Change 466980 merged by jenkins-bot:
[mediawiki/core@master] Split out new RefreshSecondaryDataUpdate class

https://gerrit.wikimedia.org/r/466980

Change 466981 merged by jenkins-bot:
[mediawiki/core@master] Make DeferredUpdates enqueue updates that failed to run when possible

https://gerrit.wikimedia.org/r/466981

Change 466981 merged by jenkins-bot:
[mediawiki/core@master] Make DeferredUpdates enqueue updates that failed to run when possible

https://gerrit.wikimedia.org/r/466981

This was reverted in 6b7ddf9c9bf / https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/497324/.

Change 497537 had a related patch set uploaded (by Aaron Schulz; owner: Aaron Schulz):
[mediawiki/core@master] Make DeferredUpdates enqueue jobs to finish failed updates

https://gerrit.wikimedia.org/r/497537

(Reflecting the patch author.)

Change 513213 had a related patch set uploaded (by Aaron Schulz; owner: Aaron Schulz):
[mediawiki/core@master] Make sure that each DataUpdate still has outer transaction scope

https://gerrit.wikimedia.org/r/513213

Not seeing this in the logs lately.

Higher level issue of re-trying LinksUpdate now tracked at T206283.

mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:08 PM