Page MenuHomePhabricator

Wikimedia\Rdbms\DBQueryError: Error 1213: Deadlock found when trying to get lock; try restarting transactionFunction: SqlBagOStuff::deleteServerObjectsExpiringBeforeQuery: DELETE FROM `objectstash` WHERE (exptime < '202302230
Closed, ResolvedPublicPRODUCTION ERROR

Description

Error
normalized_message
[{reqId}] {exception_url}   Wikimedia\Rdbms\DBQueryError: Error 1213: Deadlock found when trying to get lock; try restarting transaction
Function: SqlBagOStuff::deleteServerObjectsExpiringBefore
Query: DELETE FROM `objectstash` WHERE (exptime < '202302230
exception.trace
from /srv/mediawiki/php-1.40.0-wmf.23/includes/libs/rdbms/database/Database.php(1534)
#0 /srv/mediawiki/php-1.40.0-wmf.23/includes/libs/rdbms/database/Database.php(1518): Wikimedia\Rdbms\Database->getQueryException(string, integer, string, string)
#1 /srv/mediawiki/php-1.40.0-wmf.23/includes/libs/rdbms/database/Database.php(1492): Wikimedia\Rdbms\Database->getQueryExceptionAndLog(string, integer, string, string)
#2 /srv/mediawiki/php-1.40.0-wmf.23/includes/libs/rdbms/database/Database.php(865): Wikimedia\Rdbms\Database->reportQueryError(string, integer, string, string, boolean)
#3 /srv/mediawiki/php-1.40.0-wmf.23/includes/libs/rdbms/database/Database.php(2021): Wikimedia\Rdbms\Database->query(string, string, integer)
#4 /srv/mediawiki/php-1.40.0-wmf.23/includes/libs/rdbms/database/DBConnRef.php(119): Wikimedia\Rdbms\Database->delete(string, array, string)
#5 /srv/mediawiki/php-1.40.0-wmf.23/includes/libs/rdbms/database/DBConnRef.php(560): Wikimedia\Rdbms\DBConnRef->__call(string, array)
#6 /srv/mediawiki/php-1.40.0-wmf.23/includes/objectcache/SqlBagOStuff.php(1504): Wikimedia\Rdbms\DBConnRef->delete(string, array, string)
#7 /srv/mediawiki/php-1.40.0-wmf.23/includes/objectcache/SqlBagOStuff.php(1370): SqlBagOStuff->deleteServerObjectsExpiringBefore(Wikimedia\Rdbms\DBConnRef, integer, integer)
#8 /srv/mediawiki/php-1.40.0-wmf.23/includes/deferred/MWCallableUpdate.php(38): SqlBagOStuff->{closure}()
#9 /srv/mediawiki/php-1.40.0-wmf.23/includes/deferred/DeferredUpdates.php(473): MWCallableUpdate->doUpdate()
#10 /srv/mediawiki/php-1.40.0-wmf.23/includes/deferred/DeferredUpdates.php(398): DeferredUpdates::attemptUpdate(MWCallableUpdate, Wikimedia\Rdbms\LBFactoryMulti)
#11 /srv/mediawiki/php-1.40.0-wmf.23/includes/deferred/DeferredUpdates.php(226): DeferredUpdates::run(MWCallableUpdate, Wikimedia\Rdbms\LBFactoryMulti, Monolog\Logger, BufferingStatsdDataFactory, MediaWiki\JobQueue\JobQueueGroupFactory, string)
#12 /srv/mediawiki/php-1.40.0-wmf.23/includes/deferred/DeferredUpdatesScope.php(267): DeferredUpdates::{closure}(MWCallableUpdate, integer)
#13 /srv/mediawiki/php-1.40.0-wmf.23/includes/deferred/DeferredUpdatesScope.php(196): DeferredUpdatesScope->processStageQueue(integer, integer, Closure)
#14 /srv/mediawiki/php-1.40.0-wmf.23/includes/deferred/DeferredUpdates.php(229): DeferredUpdatesScope->processUpdates(integer, Closure)
#15 /srv/mediawiki/php-1.40.0-wmf.23/includes/deferred/DeferredUpdatesScope.php(267): DeferredUpdates::{closure}(MWCallableUpdate, integer)
#16 /srv/mediawiki/php-1.40.0-wmf.23/includes/deferred/DeferredUpdatesScope.php(196): DeferredUpdatesScope->processStageQueue(integer, integer, Closure)
#17 /srv/mediawiki/php-1.40.0-wmf.23/includes/deferred/DeferredUpdates.php(234): DeferredUpdatesScope->processUpdates(integer, Closure)
#18 /srv/mediawiki/php-1.40.0-wmf.23/includes/MediaWiki.php(1117): DeferredUpdates::doUpdates()
#19 /srv/mediawiki/php-1.40.0-wmf.23/includes/MediaWiki.php(848): MediaWiki->restInPeace()
#20 /srv/mediawiki/php-1.40.0-wmf.23/load.php(61): MediaWiki->doPostOutputShutdown()
#21 /srv/mediawiki/php-1.40.0-wmf.23/load.php(39): wfLoadMain()
#22 /srv/mediawiki/w/load.php(3): require(string)
#23 {main}
Impact

A single occurrence?

Notes

There is a database deadlock when trying to expire ResourceLoader modules from the cache. That is done as a DeferredUpdate I don't think there
is much impact, maybe it is a contention cause multiple queries trigger the expiry mechanism to kick in?

Details

Request URL
https://ko.wikipedia.org/w/load.php?lang=*&modules=*&only=*&skin=*

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

I only see a few of these in the last 7 days on logstash. Half the keys seem to be captcha related.

Krinkle triaged this task as High priority.Mar 27 2023, 6:29 AM

Change 904866 had a related patch set uploaded (by Krinkle; author: Krinkle):

[mediawiki/core@master] objectcache: Use DB lock in SQLBagOStuff purge to avoid deadlock

https://gerrit.wikimedia.org/r/904866

Change 904866 merged by jenkins-bot:

[mediawiki/core@master] objectcache: Use DB lock in SQLBagOStuff purge to avoid deadlock

https://gerrit.wikimedia.org/r/904866