Page MenuHomePhabricator
Create Task
Maniphest T369461

UserEditCountUpdate: "Cannot execute Wikimedia\Rdbms\Database::runOnTransactionIdleCallbacks critical section while session state is out of sync" triggered from some API call initiated by Commons search
Open, Needs TriagePublicPRODUCTION ERROR
Actions

Description

Cannot execute Wikimedia\Rdbms\Database::runOnTransactionIdleCallbacks critical section while session state is out of sync.

A critical section from Wikimedia\Rdbms\Database::executeQuery has failed
#0 /srv/mediawiki/php-1.43.0-wmf.12/vendor/wikimedia/request-timeout/src/CriticalSectionScope.php(44): Wikimedia\Rdbms\Database->Wikimedia\Rdbms\{closure}(567)
#1 /srv/mediawiki/php-1.43.0-wmf.12/includes/libs/rdbms/database/Database.php(634): Wikimedia\RequestTimeout\CriticalSectionScope->__destruct()
#2 /srv/mediawiki/php-1.43.0-wmf.12/includes/libs/rdbms/database/Database.php(1496): Wikimedia\Rdbms\Database->query(Object(Wikimedia\Rdbms\Query), 'MediaWiki\\Defer...')
#3 /srv/mediawiki/php-1.43.0-wmf.12/includes/libs/rdbms/database/DBConnRef.php(127): Wikimedia\Rdbms\Database->update('user', Array, Array, 'MediaWiki\\Defer...', Array)
#4 /srv/mediawiki/php-1.43.0-wmf.12/includes/libs/rdbms/database/DBConnRef.php(413): Wikimedia\Rdbms\DBConnRef->__call('update', Array)
#5 /srv/mediawiki/php-1.43.0-wmf.12/includes/libs/rdbms/querybuilder/UpdateQueryBuilder.php(335): Wikimedia\Rdbms\DBConnRef->update('user', Array, Array, 'MediaWiki\\Defer...', Array)
#6 /srv/mediawiki/php-1.43.0-wmf.12/includes/deferred/UserEditCountUpdate.php(90): Wikimedia\Rdbms\UpdateQueryBuilder->execute()
#7 /srv/mediawiki/php-1.43.0-wmf.12/includes/deferred/AutoCommitUpdate.php(47): MediaWiki\Deferred\UserEditCountUpdate->MediaWiki\Deferred\{closure}(Object(Wikimedia\Rdbms\DBConnRef), 'MediaWiki\\Defer...')
#8 /srv/mediawiki/php-1.43.0-wmf.12/includes/deferred/UserEditCountUpdate.php(107): MediaWiki\Deferred\AutoCommitUpdate->doUpdate()
#9 /srv/mediawiki/php-1.43.0-wmf.12/includes/deferred/DeferredUpdates.php(486): MediaWiki\Deferred\UserEditCountUpdate->doUpdate()
#10 /srv/mediawiki/php-1.43.0-wmf.12/includes/deferred/DeferredUpdates.php(198): MediaWiki\Deferred\DeferredUpdates::attemptUpdate(Object(MediaWiki\Deferred\UserEditCountUpdate))
#11 /srv/mediawiki/php-1.43.0-wmf.12/includes/deferred/DeferredUpdates.php(285): MediaWiki\Deferred\DeferredUpdates::run(Object(MediaWiki\Deferred\UserEditCountUpdate))
#12 /srv/mediawiki/php-1.43.0-wmf.12/includes/deferred/DeferredUpdatesScope.php(269): MediaWiki\Deferred\DeferredUpdates::MediaWiki\Deferred\{closure}(Object(MediaWiki\Deferred\UserEditCountUpdate), 2)
#13 /srv/mediawiki/php-1.43.0-wmf.12/includes/deferred/DeferredUpdatesScope.php(198): MediaWiki\Deferred\DeferredUpdatesScope->processStageQueue(2, 2, Object(Closure))
#14 /srv/mediawiki/php-1.43.0-wmf.12/includes/deferred/DeferredUpdates.php(304): MediaWiki\Deferred\DeferredUpdatesScope->processUpdates(0, Object(Closure))
#15 /srv/mediawiki/php-1.43.0-wmf.12/includes/MediaWikiEntryPoint.php(673): MediaWiki\Deferred\DeferredUpdates::doUpdates()
#16 /srv/mediawiki/php-1.43.0-wmf.12/includes/MediaWikiEntryPoint.php(496): MediaWiki\MediaWikiEntryPoint->restInPeace()
#17 /srv/mediawiki/php-1.43.0-wmf.12/includes/MediaWikiEntryPoint.php(454): MediaWiki\MediaWikiEntryPoint->doPostOutputShutdown()
#18 /srv/mediawiki/php-1.43.0-wmf.12/includes/MediaWikiEntryPoint.php(209): MediaWiki\MediaWikiEntryPoint->postOutputShutdown()
#19 /srv/mediawiki/php-1.43.0-wmf.12/api.php(44): MediaWiki\MediaWikiEntryPoint->run()
#20 /srv/mediawiki/w/api.php(3): require('/srv/mediawiki/...')
#21 {main}

Stack trace (not really useful since this is happening on shutdown):

from /srv/mediawiki/php-1.43.0-wmf.12/includes/libs/rdbms/database/Database.php(3058)
#0 /srv/mediawiki/php-1.43.0-wmf.12/includes/libs/rdbms/database/Database.php(2003): Wikimedia\Rdbms\Database->commenceCriticalSection(string)
#1 /srv/mediawiki/php-1.43.0-wmf.12/includes/libs/rdbms/loadbalancer/LoadBalancer.php(1526): Wikimedia\Rdbms\Database->runOnTransactionIdleCallbacks(integer, array)
#2 /srv/mediawiki/php-1.43.0-wmf.12/includes/libs/rdbms/lbfactory/LBFactory.php(377): Wikimedia\Rdbms\LoadBalancer->runPrimaryTransactionIdleCallbacks(string)
#3 /srv/mediawiki/php-1.43.0-wmf.12/includes/libs/rdbms/lbfactory/LBFactory.php(352): Wikimedia\Rdbms\LBFactory->executePostTransactionCallbacks()
#4 /srv/mediawiki/php-1.43.0-wmf.12/includes/exception/MWExceptionHandler.php(170): Wikimedia\Rdbms\LBFactory->rollbackPrimaryChanges(string)
#5 /srv/mediawiki/php-1.43.0-wmf.12/includes/exception/MWExceptionHandler.php(195): MWExceptionHandler::rollbackPrimaryChanges()
#6 /srv/mediawiki/php-1.43.0-wmf.12/includes/MediaWikiEntryPoint.php(500): MWExceptionHandler::rollbackPrimaryChangesAndLog(Wikimedia\Rdbms\DBUnexpectedError, string)
#7 /srv/mediawiki/php-1.43.0-wmf.12/includes/MediaWikiEntryPoint.php(454): MediaWiki\MediaWikiEntryPoint->doPostOutputShutdown()
#8 /srv/mediawiki/php-1.43.0-wmf.12/includes/MediaWikiEntryPoint.php(209): MediaWiki\MediaWikiEntryPoint->postOutputShutdown()
#9 /srv/mediawiki/php-1.43.0-wmf.12/api.php(44): MediaWiki\MediaWikiEntryPoint->run()
#10 /srv/mediawiki/w/api.php(3): require(string)
#11 {main}

281 instances in the last 7 days, all on Commons from api.php. Unfortunately on POST requests parameters are not recorded, but the referrer is always Special:Search. Nothing on the search page should trigger a POST request, so maybe this is from some gadget that does some mass operation on search results? I guess since the error is from UserEditCountUpdate, it doesn't matter much - could be anything that makes an edit.

Details

Request URL
https://commons.wikimedia.org/w/api.php

Related Objects

Event Timeline

Tgr created this task.Jul 7 2024, 3:19 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 7 2024, 3:19 PM
Tgr added a project: MediaWiki-General.Jul 7 2024, 3:23 PM

There's a bunch of similar tasks:

but they have very different stack traces, so I assume this is a problem with application code, not the RDBMS library. Not sure what's the right component for UserEditCountTracker though.

Tgr added a comment.Jul 7 2024, 3:28 PM

There is also a separate batch of errors (88 in last 7 days) with a different stack-trace-in-message:

Cannot execute Wikimedia\Rdbms\Database::runOnTransactionIdleCallbacks critical section while session state is out of sync.

A critical section from Wikimedia\Rdbms\Database::runOnTransactionIdleCallbacks has failed
#0 /srv/mediawiki/php-1.43.0-wmf.11/vendor/wikimedia/request-timeout/src/CriticalSectionScope.php(44): Wikimedia\Rdbms\Database->Wikimedia\Rdbms\{closure}(345)
#1 /srv/mediawiki/php-1.43.0-wmf.11/includes/libs/rdbms/loadbalancer/LoadBalancer.php(1526): Wikimedia\RequestTimeout\CriticalSectionScope->__destruct()
#2 /srv/mediawiki/php-1.43.0-wmf.11/includes/libs/rdbms/lbfactory/LBFactory.php(377): Wikimedia\Rdbms\LoadBalancer->runPrimaryTransactionIdleCallbacks('Wikimedia\\Rdbms...')
#3 /srv/mediawiki/php-1.43.0-wmf.11/includes/libs/rdbms/lbfactory/LBFactory.php(334): Wikimedia\Rdbms\LBFactory->executePostTransactionCallbacks()
#4 /srv/mediawiki/php-1.43.0-wmf.11/includes/deferred/DeferredUpdatesScopeMediaWikiStack.php(128): Wikimedia\Rdbms\LBFactory->commitPrimaryChanges('Wikibase\\Repo\\C...')
#5 /srv/mediawiki/php-1.43.0-wmf.11/includes/deferred/DeferredUpdates.php(488): MediaWiki\Deferred\DeferredUpdatesScopeMediaWikiStack->onRunUpdateEnd(Object(Wikibase\Repo\Content\DataUpdateAdapter))
#6 /srv/mediawiki/php-1.43.0-wmf.11/includes/deferred/RefreshSecondaryDataUpdate.php(112): MediaWiki\Deferred\DeferredUpdates::attemptUpdate(Object(Wikibase\Repo\Content\DataUpdateAdapter))
#7 /srv/mediawiki/php-1.43.0-wmf.11/includes/deferred/DeferredUpdates.php(486): MediaWiki\Deferred\RefreshSecondaryDataUpdate->doUpdate()
#8 /srv/mediawiki/php-1.43.0-wmf.11/includes/deferred/DeferredUpdates.php(198): MediaWiki\Deferred\DeferredUpdates::attemptUpdate(Object(MediaWiki\Deferred\RefreshSecondaryDataUpdate))
#9 /srv/mediawiki/php-1.43.0-wmf.11/includes/deferred/DeferredUpdates.php(285): MediaWiki\Deferred\DeferredUpdates::run(Object(MediaWiki\Deferred\RefreshSecondaryDataUpdate))
#10 /srv/mediawiki/php-1.43.0-wmf.11/includes/deferred/DeferredUpdatesScope.php(266): MediaWiki\Deferred\DeferredUpdates::MediaWiki\Deferred\{closure}(Object(MediaWiki\Deferred\RefreshSecondaryDataUpdate), 2)
#11 /srv/mediawiki/php-1.43.0-wmf.11/includes/deferred/DeferredUpdatesScope.php(198): MediaWiki\Deferred\DeferredUpdatesScope->processStageQueue(2, 2, Object(Closure))
#12 /srv/mediawiki/php-1.43.0-wmf.11/includes/deferred/DeferredUpdates.php(304): MediaWiki\Deferred\DeferredUpdatesScope->processUpdates(0, Object(Closure))
#13 /srv/mediawiki/php-1.43.0-wmf.11/includes/MediaWikiEntryPoint.php(673): MediaWiki\Deferred\DeferredUpdates::doUpdates()
#14 /srv/mediawiki/php-1.43.0-wmf.11/includes/MediaWikiEntryPoint.php(496): MediaWiki\MediaWikiEntryPoint->restInPeace()
#15 /srv/mediawiki/php-1.43.0-wmf.11/includes/MediaWikiEntryPoint.php(454): MediaWiki\MediaWikiEntryPoint->doPostOutputShutdown()
#16 /srv/mediawiki/php-1.43.0-wmf.11/includes/MediaWikiEntryPoint.php(209): MediaWiki\MediaWikiEntryPoint->postOutputShutdown()
#17 /srv/mediawiki/php-1.43.0-wmf.11/api.php(44): MediaWiki\MediaWikiEntryPoint->run()
#18 /srv/mediawiki/w/api.php(3): require('/srv/mediawiki/...')
#19 {main}

Those all seem to be coming from the Wikidata wbsetlabel API.
(example)

Krinkle moved this task from Untriaged to Jul 2024 on the Wikimedia-production-error board.Jul 25 2024, 3:23 PM
Krinkle edited projects, added MediaWiki-Search, Discovery-Search; removed MediaWiki-General, MediaWiki-libs-Rdbms.
Krinkle subscribed.

Indeed, Rdbms is detecting this and reporting it. It's not causing it. It's not clear which component is causing it, but they seem to have Search as common referer, perhaps we can investigate from there to see what it is?

dr0ptp4kt added subscribers: MarkTraceur, dr0ptp4kt.Jul 29 2024, 3:33 PM

@MarkTraceur is this possibly for media search for your team?

dr0ptp4kt moved this task from needs triage to watching / waiting on the Discovery-Search board.Jul 29 2024, 3:33 PM
jnuche mentioned this in T375615: Increasing Wikimedia\Rdbms errors: "critical section while session state is out of sync".Sep 25 2024, 11:58 AM
Aklapper mentioned this in T403058: Wikimedia\Rdbms\DBTransactionError: Transaction round stage must be 'cursory' (not 'within-rollback') / (not 'within-rollback-callbacks').Aug 27 2025, 1:30 PM
TheDJ moved this task from Backlog to Search Core on the MediaWiki-Search board.Sep 6 2025, 7:52 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits