Page MenuHomePhabricator

ClearWatchlistNotifications fails with "Lock wait timeout exceeded" on watchlist.wl_notificationtimestamp
Closed, ResolvedPublicPRODUCTION ERROR

Description

Error

Request ID: W5-PowrAADYAAAaR0bAAAACD

message
A database query error has occurred. [..]
Query: UPDATE  `watchlist` SET wl_notificationtimestamp = NULL WHERE wl_user = '###' AND wl_notificationtimestamp IN ('##############', ..., '##############')  AND (wl_notificationtimestamp < '##############)
Function: ClearWatchlistNotificationsJob::run
Error: 1205 Lock wait timeout exceeded; try restarting transaction (10.192.48.114)
stacktrace
#2 /srv/mediawiki/php-1.32.0-wmf.20/includes/libs/rdbms/database/Database.php(2074): Wikimedia\Rdbms\Database->query(string, string)
#3 /srv/mediawiki/php-1.32.0-wmf.20/includes/jobqueue/jobs/ClearWatchlistNotificationsJob.php(75): Wikimedia\Rdbms\Database->update(string, array, array, string)
#4 /srv/mediawiki/php-1.32.0-wmf.20/includes/watcheditem/WatchedItemStore.php(785): ClearWatchlistNotificationsJob->run()
#5 /srv/mediawiki/php-1.32.0-wmf.20/includes/deferred/MWCallableUpdate.php(34): Closure$WatchedItemStore::resetAllNotificationTimestampsForUser()
#6 /srv/mediawiki/php-1.32.0-wmf.20/includes/deferred/DeferredUpdates.php(268): MWCallableUpdate->doUpdate()
#7 /srv/mediawiki/php-1.32.0-wmf.20/includes/deferred/DeferredUpdates.php(214): DeferredUpdates::runUpdate(MWCallableUpdate, Wikimedia\Rdbms\LBFactoryMulti, string, integer)
#8 /srv/mediawiki/php-1.32.0-wmf.20/includes/deferred/DeferredUpdates.php(134): DeferredUpdates::execute(array, string, integer)
#9 /srv/mediawiki/php-1.32.0-wmf.20/includes/MediaWiki.php(914): DeferredUpdates::doUpdates(string)
#10 /srv/mediawiki/php-1.32.0-wmf.20/includes/MediaWiki.php(734): MediaWiki->restInPeace(string, boolean)
#11 [internal function]: Closure$MediaWiki::doPostOutputShutdown()

Notes

Happens from multiple wikis, immediately after submitting a form on Special:Watchlist. The update runs after the user receives the page response, so the error is unable to show itself to the user, which means they don't know that it failed.

See also: T134613: updateNotificationTimestamp() should batch queries

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Krinkle renamed this task from ClearWatchlistNotifications query fails with "Lock wait timeout exceeded" on watchlist.wl_notificationtimestamp) to ClearWatchlistNotifications fails with "Lock wait timeout exceeded" on watchlist.wl_notificationtimestamp.Sep 17 2018, 4:45 PM
mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:08 PM

I would say that this is fixed with the patch set. The update now used the primary key and batching which makes it less likly to run into a deadlock situation

T207941

Krinkle assigned this task to Catrope.