Page MenuHomePhabricator

MessageGroupStats::withLock: got COMMIT while atomic sections MediaWiki\Storage\PageUpdater::getAtomicSectionUpdate are still open
Closed, ResolvedPublicPRODUCTION ERROR

Description

Summary: Translation statistics failed to update when translation strings were renamed using ReplaceText extension.

Error:

[2020-03-15 12:51:57] exception.ERROR: [c0085dc3ce9b596e9e47c6ed] [no req]   Wikimedia\Rdbms\DBUnexpectedError from line 4262 of /srv/mediawiki/tags/2020-03-11_14:19:45/includes/libs/rdbms/database/Database.php: MessageGroupStats::withLock: got COMMIT while atomic sections MediaWiki\Storage\PageUpdater::getAtomicSectionUpdate are still open {"exception":"[object] (Wikimedia\\Rdbms\\DBUnexpectedError(code: 0): MessageGroupStats::withLock: got COMMIT while atomic sections MediaWiki\\Storage\\PageUpdater::getAtomicSectionUpdate are still open at /srv/mediawiki/tags/2020-03-11_14:19:45/includes/libs/rdbms/database/Database.php:4262)

TODO: Add more info after research

We've been seeing this in our logs for a few days now.

Details

Stack Trace
#0 /srv/mediawiki/tags/2020-03-11_14:19:45/includes/libs/rdbms/database/DBConnRef.php(68): Wikimedia\\Rdbms\\Database->commit('MessageGroupSta...', 'flush')                                                                                                                  
#1 /srv/mediawiki/tags/2020-03-11_14:19:45/includes/libs/rdbms/database/DBConnRef.php(649): Wikimedia\\Rdbms\\DBConnRef->__call('commit', Array)                                                                                                                              
#2 /srv/mediawiki/tags/2020-03-11_14:19:45/extensions/Translate/utils/MessageGroupStats.php(641): Wikimedia\\Rdbms\\DBConnRef->commit('MessageGroupSta...', 'flush')                                                                                                          
#3 /srv/mediawiki/tags/2020-03-11_14:19:45/extensions/Translate/utils/MessageGroupStats.php(627): MessageGroupStats::{closure}()
#4 /srv/mediawiki/tags/2020-03-11_14:19:45/extensions/Translate/utils/MessageGroupStats.php(213): MessageGroupStats::queueUpdates(0)
#5 /srv/mediawiki/tags/2020-03-11_14:19:45/extensions/Translate/utils/MessageGroupStats.php(183): MessageGroupStats::internalClearGroups('uk', Array)                                                                                                                         
#6 /srv/mediawiki/tags/2020-03-11_14:19:45/extensions/Translate/TranslateEditAddons.php(234): MessageGroupStats::clear(Object(MessageHandle))                                                                                                                                 
#7 /srv/mediawiki/tags/2020-03-11_14:19:45/includes/Hooks.php(174): TranslateEditAddons::onSave(Object(WikiPage), Object(User), Object(WikitextContent), 'Text replacemen...', 4, NULL, NULL, 6, Object(Revision), Object(Status), false, 0)                                  
#8 /srv/mediawiki/tags/2020-03-11_14:19:45/includes/Hooks.php(202): Hooks::callHook('PageContentSave...', Array, Array, NULL)
#9 /srv/mediawiki/tags/2020-03-11_14:19:45/includes/Storage/PageUpdater.php(1249): Hooks::run('PageContentSave...', Array)
#10 /srv/mediawiki/tags/2020-03-11_14:19:45/includes/libs/rdbms/database/Database.php(4179): MediaWiki\\Storage\\PageUpdater->MediaWiki\\Storage\\{closure}(Object(Wikimedia\\Rdbms\\DatabaseMysqli), 'MediaWiki\\\\Stora...')                                                
#11 /srv/mediawiki/tags/2020-03-11_14:19:45/includes/libs/rdbms/database/DBConnRef.php(68): Wikimedia\\Rdbms\\Database->doAtomicSection('MediaWiki\\\\Stora...', Object(Closure))                                                                                             
#12 /srv/mediawiki/tags/2020-03-11_14:19:45/includes/libs/rdbms/database/DBConnRef.php(641): Wikimedia\\Rdbms\\DBConnRef->__call('doAtomicSection', Array)                                                                                                                    
#13 /srv/mediawiki/tags/2020-03-11_14:19:45/includes/deferred/AtomicSectionUpdate.php(39): Wikimedia\\Rdbms\\DBConnRef->doAtomicSection('MediaWiki\\\\Stora...', Object(Closure))                                                                                             
#14 /srv/mediawiki/tags/2020-03-11_14:19:45/includes/deferred/DeferredUpdates.php(417): AtomicSectionUpdate->doUpdate()
#15 /srv/mediawiki/tags/2020-03-11_14:19:45/includes/deferred/DeferredUpdates.php(296): DeferredUpdates::attemptUpdate(Object(AtomicSectionUpdate), Object(Wikimedia\\Rdbms\\LBFactorySimple))                                                                                
#16 /srv/mediawiki/tags/2020-03-11_14:19:45/includes/deferred/DeferredUpdates.php(233): DeferredUpdates::run(Object(AtomicSectionUpdate), Object(Wikimedia\\Rdbms\\LBFactorySimple), Object(Monolog\\Logger), Object(BufferingStatsdDataFactory), 'cli')                      
#17 /srv/mediawiki/tags/2020-03-11_14:19:45/includes/deferred/DeferredUpdates.php(146): DeferredUpdates::handleUpdateQueue(Array, 'run', 2)                                                                                                                                   
#18 /srv/mediawiki/tags/2020-03-11_14:19:45/includes/jobqueue/JobRunner.php(377): DeferredUpdates::doUpdates()
#19 /srv/mediawiki/tags/2020-03-11_14:19:45/includes/jobqueue/JobRunner.php(333): JobRunner->doExecuteJob(Object(ReplaceTextJob))
#20 /srv/mediawiki/tags/2020-03-11_14:19:45/includes/jobqueue/JobRunner.php(244): JobRunner->executeJob(Object(ReplaceTextJob))
#21 /srv/mediawiki/tags/2020-03-11_14:19:45/maintenance/runJobs.php(92): JobRunner->run(Array)
#22 /srv/mediawiki/tags/2020-03-11_14:19:45/maintenance/doMaintenance.php(107): RunJobs->execute()
#23 /srv/mediawiki/tags/2020-03-11_14:19:45/maintenance/runJobs.php(129): require_once('/srv/mediawiki/...')

Event Timeline

abi_ triaged this task as Medium priority.Mar 17 2020, 7:29 AM
abi_ created this task.
abi_ created this object with edit policy "All Users".

Should also check for recent changes in ReplaceText extension.

Change 592640 had a related patch set uploaded (by Nikerabbit; owner: Nikerabbit):
[mediawiki/extensions/Translate@master] MessageGroupStats: Avoid COMMIT while atomic section is open

https://gerrit.wikimedia.org/r/592640

Turns out this can be fixed in Translate, even though it might be a core change that caused this to start happening.

Change 592640 merged by jenkins-bot:
[mediawiki/extensions/Translate@master] MessageGroupStats: Avoid COMMIT while atomic section is open

https://gerrit.wikimedia.org/r/592640

This has been deployed on translatewiki.net. We will need to monitor this for a few days before marking it as done.

Tested this with a manual rename using Special:ReplaceText, and did not notice any issues. Also haven't seen any related logs for this in a while.