BagOStuff::trackDuplicateKeys causes a "MWCallableUpdate::doUpdate: transaction round ..." during JobRunner::executeJob
Closed, ResolvedPublic
Actions

Description

MediaWiki	1.30.0-alpha (3d26a96)
PHP	7.1.1 (apache2handler)
MariaDB	10.1.21-MariaDB
ICU	57.1

[Bug56269] Exception thrown with an uncommited database transaction: [d8d3f093942c3e9b92a83e61] /mw-master/index.php/Special:Browse/:Recurring-5Fevents-2F41   Wikimedia\Rdbms\DBTransactionError from line 200 of ...\includes\libs\rdbms\lbfactory\LBFactory.php: MWCallableUpdate::doUpdate: transaction round 'SMW\MediaWiki\Jobs\UpdateJob::run' already started.
#0 ...\includes\deferred\DeferredUpdates.php(265): Wikimedia\Rdbms\LBFactory->beginMasterChanges(string)
#1 ...\includes\deferred\DeferredUpdates.php(228): DeferredUpdates::runUpdate(MWCallableUpdate, Wikimedia\Rdbms\LBFactorySimple, integer)
#2 ...\includes\deferred\DeferredUpdates.php(136): DeferredUpdates::execute(array, string, integer)
#3 ...\includes\deferred\DeferredUpdates.php(95): DeferredUpdates::doUpdates(string)
#4 ...\includes\deferred\DeferredUpdates.php(119): DeferredUpdates::addUpdate(MWCallableUpdate, integer)
#5 ...\includes\libs\objectcache\BagOStuff.php(216): DeferredUpdates::addCallableUpdate(Closure)
#6 ...\includes\libs\objectcache\BagOStuff.php(183): BagOStuff->trackDuplicateKeys(string)
#7 ...\includes\parser\Preprocessor.php(110): BagOStuff->get(string)
#8 ...\includes\parser\Preprocessor_DOM.php(152): Preprocessor->cacheGetTree(string, integer)
#9 ...\includes\parser\Parser.php(2889): Preprocessor_DOM->preprocessToObj(string, integer)
#10 ...\includes\parser\Parser.php(4121): Parser->preprocessToDom(string)
#11 ...\includes\parser\Parser.php(1339): Parser->formatHeadings(string, string, boolean)
#12 ...\includes\parser\Parser.php(451): Parser->internalParse(string)
#13 ...\includes\content\WikitextContent.php(330): Parser->parse(string, Title, ParserOptions, boolean, boolean, integer)
#14 ...\includes\content\AbstractContent.php(497): WikitextContent->fillParserOutput(Title, integer, ParserOptions, boolean, ParserOutput)
#15 ...\extensions\SemanticMediaWiki\includes\ContentParser.php(203): AbstractContent->getParserOutput(Title, integer, ParserOptions, boolean)
#16 ...\extensions\SemanticMediaWiki\includes\ContentParser.php(156): SMW\ContentParser->fetchFromContent()
#17 ...\extensions\SemanticMediaWiki\src\MediaWiki\Jobs\UpdateJob.php(139): SMW\ContentParser->parse()
#18 ...\extensions\SemanticMediaWiki\src\MediaWiki\Jobs\UpdateJob.php(112): SMW\MediaWiki\Jobs\UpdateJob->needToParsePageContentBeforeUpdate()
#19 ...\extensions\SemanticMediaWiki\src\MediaWiki\Jobs\UpdateJob.php(80): SMW\MediaWiki\Jobs\UpdateJob->doPrepareForUpdate()
#20 ...\extensions\SemanticMediaWiki\src\MediaWiki\Jobs\UpdateJob.php(61): SMW\MediaWiki\Jobs\UpdateJob->doUpdate()
#21 ...\includes\jobqueue\JobRunner.php(289): SMW\MediaWiki\Jobs\UpdateJob->run()
#22 ...\includes\jobqueue\JobRunner.php(189): JobRunner->executeJob(SMW\MediaWiki\Jobs\UpdateJob, Wikimedia\Rdbms\LBFactorySimple, BufferingStatsdDataFactory, integer)
#23 ...\includes\MediaWiki.php(973): JobRunner->run(array)
#24 ...\includes\MediaWiki.php(959): MediaWiki->triggerSyncJobs(integer, MediaWiki\Logger\LegacyLogger)
#25 ...\includes\MediaWiki.php(911): MediaWiki->triggerJobs()
#26 ...\includes\MediaWiki.php(731): MediaWiki->restInPeace(string)
#27 ...\includes\MediaWiki.php(750): MediaWiki->{closure}()
#28 ...\includes\MediaWiki.php(554): MediaWiki->doPostOutputShutdown(string)
#29 ...\index.php(43): MediaWiki->run()

Issue

The issue appears with BagOStuff::trackDuplicateKeys invoking DeferredUpdates::addCallableUpdate while a job is active and causes a "MWCallableUpdate::doUpdate: transaction round 'SMW\MediaWiki\Jobs\UpdateJob::run' already started."

... and before someone (as it has been so in the past) to declare this to be a SMW issue and closes the task before investigating, here is an example that doesn't involve SMW at all.

[Bug56269] Exception thrown with an uncommited database transaction: [dcf60ceb63f75441b711c08b] /mw-master/index.php?search=Lorem%20ipsum-1d&title=Special%3ASearch   Wikimedia\Rdbms\DBTransactionError from line 200 of ...\includes\libs\rdbms\lbfactory\LBFactory.php: MWCallableUpdate::doUpdate: transaction round 'RefreshLinksJob::run' already started.
#0 ...\includes\deferred\DeferredUpdates.php(265): Wikimedia\Rdbms\LBFactory->beginMasterChanges(string)
#1 ...\includes\deferred\DeferredUpdates.php(228): DeferredUpdates::runUpdate(MWCallableUpdate, Wikimedia\Rdbms\LBFactorySimple, integer)
#2 ...\includes\deferred\DeferredUpdates.php(136): DeferredUpdates::execute(array, string, integer)
#3 ...\includes\deferred\DeferredUpdates.php(95): DeferredUpdates::doUpdates(string)
#4 ...\includes\deferred\DeferredUpdates.php(119): DeferredUpdates::addUpdate(MWCallableUpdate, integer)
#5 ...\includes\libs\objectcache\BagOStuff.php(216): DeferredUpdates::addCallableUpdate(Closure)
#6 ...\includes\libs\objectcache\BagOStuff.php(183): BagOStuff->trackDuplicateKeys(string)
#7 ...\includes\parser\Preprocessor.php(110): BagOStuff->get(string)
#8 ...\includes\parser\Preprocessor_DOM.php(152): Preprocessor->cacheGetTree(string, integer)
#9 ...\includes\parser\Parser.php(2889): Preprocessor_DOM->preprocessToObj(string, integer)
#10 ...\includes\parser\Parser.php(4121): Parser->preprocessToDom(string)
#11 ...\includes\parser\Parser.php(1339): Parser->formatHeadings(string, string, boolean)
#12 ...\includes\parser\Parser.php(451): Parser->internalParse(string)
#13 ...\includes\StubObject.php(113): Parser->parse(string, Title, ParserOptions, boolean, boolean, integer)
#14 ...\includes\StubObject.php(139): StubObject->_call(string, array)
#15 ...\includes\content\WikitextContent.php(330): StubObject->__call(string, array)
#16 ...\includes\content\AbstractContent.php(497): WikitextContent->fillParserOutput(Title, integer, ParserOptions, boolean, ParserOutput)
#17 ...\includes\jobqueue\jobs\RefreshLinksJob.php(227): AbstractContent->getParserOutput(Title, integer, ParserOptions, boolean)
#18 ...\includes\jobqueue\jobs\RefreshLinksJob.php(122): RefreshLinksJob->runForTitle(Title)
#19 ...\includes\jobqueue\JobRunner.php(289): RefreshLinksJob->run()
#20 ...\includes\jobqueue\JobRunner.php(189): JobRunner->executeJob(RefreshLinksJob, Wikimedia\Rdbms\LBFactorySimple, BufferingStatsdDataFactory, integer)
#21 ...\includes\MediaWiki.php(973): JobRunner->run(array)
#22 ...\includes\MediaWiki.php(959): MediaWiki->triggerSyncJobs(integer, MediaWiki\Logger\LegacyLogger)
#23 ...\includes\MediaWiki.php(911): MediaWiki->triggerJobs()
#24 ...\includes\MediaWiki.php(731): MediaWiki->restInPeace(string)
#25 ...\includes\MediaWiki.php(750): MediaWiki->{closure}()
#26 ...\includes\MediaWiki.php(554): MediaWiki->doPostOutputShutdown(string)
#27 ...\index.php(43): MediaWiki->run()

This may or may not be related to T154438.

SUMMARY OF T165714, T100085, and T154425

All three bugs are related to jobs (background tasks managed asynchronously) and deferred updates (also called deferrable updates, tasks postponed to the end of the request). Some deferred updates can be transformed to jobs. Jobs are usually pushed in the job queue during normal operations, but some jobs are only pushed at a latter time, these are called lazy jobs (short for lazily-pushed jobs). At the end of a MediaWiki request on the Web, deferred updates are executed, then some jobs are executed if the parameter $wgJobRunRate is not 0, and lazy jobs are pushed to the job queue.

T165714 (duplicates: T154439) (affects: MW 1.28, 1.29, 1.30) Some jobs throw an exception when executed on the Web due to interactions between jobs and deferred updates linked to database transactions; these jobs are never executed and remain in the job queue until deleted. Bug trigerred when $wgJobRunRate > 0 (default) and the job queue is JobQueueDB (default). Fixed by https://gerrit.wikimedia.org/r/#/c/356120/ on 1.28, 1.29, 1.30.

T100085 (duplicates: T154427) (affects: MW: 1.27, 1.28, 1.29, 1.30) Some lazy jobs added by jobs executed on the Web are not pushed and this triggers an error in the logs; these jobs are never added in the job queue, hence never executed. Bug trigerred when $wgJobRunRate > 0 (default). Fixed by https://gerrit.wikimedia.org/r/#/c/356120/ on 1.27, 1.28, 1.29, 1.30. The fix from T154425 improves the resolution. Possibly other root causes given it happened on Translatewiki and Wikimedia (where $wgJobRunRate is 0).

T154425 (duplicates: T154438, T157679, T166867, T153849) (affects: MW 1.28, 1.29, 1.30) Some deferred updates are not executed when there exists an EnqueueableDataUpdate (a specific deferrable update, which can be transformed to a job) linked to database transactions; this happens mainly during delete, restore, move operations, possibly some others; these deferred updates are not executed. Bug trigerred when the job queue is JobQueueDB (default). Fixed by https://gerrit.wikimedia.org/r/#/c/356619/ and https://gerrit.wikimedia.org/r/#/c/357966/ in 1.28, 1.29, 1.30.

Details

Subject	Repo	Branch	Lines +/-
Better handling of jobs execution in post-connection shutdown	mediawiki/core	REL1_28	+4 -10
Better handling of jobs execution in post-connection shutdown	mediawiki/core	master	+4 -10
Better handling of jobs execution in post-connection shutdown	mediawiki/core	REL1_29	+4 -10

Customize query in gerrit

Revisions and Commits

rMW MediaWiki
	rMWff342b725a07 Make DeferredUpdates detect LBFactory transaction rounds

Related Objects

Mentioned In: T154438: Special:MovePage throws MWCallableUpdate::doUpdate: Cannot flush snapshot because writes are pending (JobQueueDB::doBatchPush, JobQueueDB::doBatchPush)
T154439: AutoCommitUpdate::doUpdate (Title->invalidateCache) causes Exception thrown with an uncommited database transaction
T100085: PHP Notice: JobQueueGroup::__destruct: 1 buffered job(s) never inserted
T154425: Delete action throws a DBUnexpectedError with "MWCallableUpdate::doUpdate: Cannot flush snapshot because writes are pending (JobQueueDB::doBatchPush)"
Mentioned Here: T153849: Deleted pages' creation entry shows up in Special:RecentChanges as a redlink since MW 1.28
T154427: ErrorException with "PHP Notice: JobQueueGroup::__destruct: 1 buffered job ... HTMLCacheUpdateJob never inserted"
T154439: AutoCommitUpdate::doUpdate (Title->invalidateCache) causes Exception thrown with an uncommited database transaction
T166867: DBUnexpectedError from line 2877 of Database.php: MWCallableUpdate::doUpdate: Cannot flush snapshot because writes are pending (JobQueueDB::doBatchPush).
T100085: PHP Notice: JobQueueGroup::__destruct: 1 buffered job(s) never inserted
T154425: Delete action throws a DBUnexpectedError with "MWCallableUpdate::doUpdate: Cannot flush snapshot because writes are pending (JobQueueDB::doBatchPush)"
T157679: Exception thrown with an uncommited database transaction: MWCallableUpdate::doUpdate: Flush failed on server(s)
T154438: Special:MovePage throws MWCallableUpdate::doUpdate: Cannot flush snapshot because writes are pending (JobQueueDB::doBatchPush, JobQueueDB::doBatchPush)

Event Timeline

mwjames created this task.May 19 2017, 6:59 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 19 2017, 6:59 AM

mwjames updated the task description. (Show Details)May 19 2017, 7:09 AM

mwjames updated the task description. (Show Details)May 19 2017, 7:33 AM

Looks like a logic error in Semantic MediaWiki, not anything in MediaWiki core or RDBMS.

[Bug56269] Exception thrown with an uncommited database transaction:
[d8d3f093942c3e9b92a83e61] /mw-master/index.php/Special:Browse/:Recurring-5Fevents-2F41
Wikimedia\Rdbms\DBTransactionError from line 200 of includes\libs\rdbms\lbfactory\LBFactory.php: MWCallableUpdate::doUpdate:
transaction round 'SMW\MediaWiki\Jobs\UpdateJob::run' already started.

Nikerabbit subscribed.May 24 2017, 11:21 AM

Krinkle moved this task from Untriaged to Usage problem on the MediaWiki-libs-Rdbms board.May 24 2017, 11:25 AM

Looks like a logic error in Semantic MediaWiki, not anything in MediaWiki core or RDBMS.

You did read the second trace, rigtht? Same symptom and issue and is caused by the RefreshLinksJob.

[Bug56269] Exception thrown with an uncommited database transaction: [dcf60ceb63f75441b711c08b] /mw-master/index.php?search=Lorem%20ipsum-1d&title=Special%3ASearch   Wikimedia\Rdbms\DBTransactionError from line 200 of ...\includes\libs\rdbms\lbfactory\LBFactory.php: MWCallableUpdate::doUpdate: transaction round 'RefreshLinksJob::run' already started.
#0 ...\includes\deferred\DeferredUpdates.php(265): Wikimedia\Rdbms\LBFactory->beginMasterChanges(string)
#1 ...\includes\deferred\DeferredUpdates.php(228): DeferredUpdates::runUpdate(MWCallableUpdate, Wikimedia\Rdbms\LBFactorySimple, integer)
#2 ...\includes\deferred\DeferredUpdates.php(136): DeferredUpdates::execute(array, string, integer)
#3 ...\includes\deferred\DeferredUpdates.php(95): DeferredUpdates::doUpdates(string)
#4 ...\includes\deferred\DeferredUpdates.php(119): DeferredUpdates::addUpdate(MWCallableUpdate, integer)
#5 ...\includes\libs\objectcache\BagOStuff.php(216): DeferredUpdates::addCallableUpdate(Closure)
#6 ...\includes\libs\objectcache\BagOStuff.php(183): BagOStuff->trackDuplicateKeys(string)
#7 ...\includes\parser\Preprocessor.php(110): BagOStuff->get(string)
#8 ...\includes\parser\Preprocessor_DOM.php(152): Preprocessor->cacheGetTree(string, integer)
#9 ...\includes\parser\Parser.php(2889): Preprocessor_DOM->preprocessToObj(string, integer)
#10 ...\includes\parser\Parser.php(4121): Parser->preprocessToDom(string)
#11 ...\includes\parser\Parser.php(1339): Parser->formatHeadings(string, string, boolean)
#12 ...\includes\parser\Parser.php(451): Parser->internalParse(string)
#13 ...\includes\StubObject.php(113): Parser->parse(string, Title, ParserOptions, boolean, boolean, integer)
#14 ...\includes\StubObject.php(139): StubObject->_call(string, array)
#15 ...\includes\content\WikitextContent.php(330): StubObject->__call(string, array)
#16 ...\includes\content\AbstractContent.php(497): WikitextContent->fillParserOutput(Title, integer, ParserOptions, boolean, ParserOutput)
#17 ...\includes\jobqueue\jobs\RefreshLinksJob.php(227): AbstractContent->getParserOutput(Title, integer, ParserOptions, boolean)
#18 ...\includes\jobqueue\jobs\RefreshLinksJob.php(122): RefreshLinksJob->runForTitle(Title)
#19 ...\includes\jobqueue\JobRunner.php(289): RefreshLinksJob->run()
#20 ...\includes\jobqueue\JobRunner.php(189): JobRunner->executeJob(RefreshLinksJob, Wikimedia\Rdbms\LBFactorySimple, BufferingStatsdDataFactory, integer)
#21 ...\includes\MediaWiki.php(973): JobRunner->run(array)
#22 ...\includes\MediaWiki.php(959): MediaWiki->triggerSyncJobs(integer, MediaWiki\Logger\LegacyLogger)
#23 ...\includes\MediaWiki.php(911): MediaWiki->triggerJobs()
#24 ...\includes\MediaWiki.php(731): MediaWiki->restInPeace(string)
#25 ...\includes\MediaWiki.php(750): MediaWiki->{closure}()
#26 ...\includes\MediaWiki.php(554): MediaWiki->doPostOutputShutdown(string)
#27 ...\index.php(43): MediaWiki->run()

I see.

The problem is likely that the duplicate key tracking is using an async handler, with the expectation of it being queued and run at an appropiate time later, by MediaWiki. However, because we're already post-shutdown and processing a job, it runs immediately. And since deferred updates should not be nested, they are given a clean transaction ticket - which fails, since a transaction is already active.

This combined with the fact that jobs are run on page views, which is after shutdown, so the job runs immediately instead of being queued first until the job finishes.

@aaron might know more. The basic pattern is basically: A job is run on index.php, and as part of that job at some point an innocent call to $cache->get() is made, which triggers duplicate key tracking, which reports the duplicate asynchronously with its Psr Logger.

That async function is queued as a deferred update and runs immediately, which then has DeferredUpdates::execute() call getEmptyTransactionTicket() and eventually beginMasterChanges().

As far as I know we have logic in place to ensure that when we're in "immediate" mode, queuing an update will add it to the list, not run it nested inside another update. However my guess is that it doesn't account for the fact that a job may be running (as opposed to an update). Presumably this doesn't affect maintenance scripts and cronned job runners since they run jobs before "shutdown" happens, not after?

Krinkle added a project: Performance-Team.May 24 2017, 11:53 AM

Krinkle moved this task from Usage problem to Error handling on the MediaWiki-libs-Rdbms board.

Krinkle moved this task from Error handling to Usage problem on the MediaWiki-libs-Rdbms board.

Krinkle added a project: MediaWiki-Core-JobQueue.

• Gilles moved this task from Inbox, needs triage to To-do: Goals prioritized current Quarter on the Performance-Team board.May 26 2017, 4:25 PM

aaron triaged this task as Medium priority.May 26 2017, 5:05 PM

That async function is queued as a deferred update and runs immediately, which then has DeferredUpdates::execute() call getEmptyTransactionTicket() and eventually beginMasterChanges().

Just found another example on JobQueueDB::claimRandom where BagOStuff::trackDuplicateKeys is breaking the transaction flow.

Title::getRestrictionTypes: applicable restrictions to [[Lorem ipsum-2ca]] are {edit,move}
Job with hash 't0ui7t0cg0b9prp29q7plzaevrijg70' is a duplicate.
Job with hash '2adp8b9bojlq6h77eift6wj2okhu5wy' is a duplicate.
[exception] [44d8e1cb3d4940a909b05512] /mw-master/index.php?title=Lorem_ipsum-2ca&action=submit   Wikimedia\Rdbms\DBUnexpectedError from line 2947 of ...\mw-master\includes\libs\rdbms\database\Database.php: MWCallableUpdate::doUpdate: Cannot flush snapshot because writes are pending (JobQueueDB::claimRandom).
#0 ...\mw-master\includes\libs\rdbms\loadbalancer\LoadBalancer.php(1200): Wikimedia\Rdbms\Database->flushSnapshot(string)
#1 [internal function]: Wikimedia\Rdbms\LoadBalancer->Wikimedia\Rdbms\{closure}(Wikimedia\Rdbms\DatabaseMysqli)
#2 ...\mw-master\includes\libs\rdbms\loadbalancer\LoadBalancer.php(1498): call_user_func_array(Closure, array)
#3 ...\mw-master\includes\libs\rdbms\loadbalancer\LoadBalancer.php(1207): Wikimedia\Rdbms\LoadBalancer->forEachOpenMasterConnection(Closure)
#4 [internal function]: Wikimedia\Rdbms\LoadBalancer->beginMasterChanges(string)
#5 ...\mw-master\includes\libs\rdbms\lbfactory\LBFactory.php(183): call_user_func_array(array, array)
#6 [internal function]: Wikimedia\Rdbms\LBFactory->Wikimedia\Rdbms\{closure}(Wikimedia\Rdbms\LoadBalancer, string, array)
#7 ...\mw-master\includes\libs\rdbms\lbfactory\LBFactorySimple.php(149): call_user_func_array(Closure, array)
#8 ...\mw-master\includes\libs\rdbms\lbfactory\LBFactory.php(185): Wikimedia\Rdbms\LBFactorySimple->forEachLB(Closure, array)
#9 ...\mw-master\includes\libs\rdbms\lbfactory\LBFactory.php(207): Wikimedia\Rdbms\LBFactory->forEachLBCallMethod(string, array)
#10 ...\mw-master\includes\deferred\DeferredUpdates.php(265): Wikimedia\Rdbms\LBFactory->beginMasterChanges(string)
#11 ...\mw-master\includes\deferred\DeferredUpdates.php(228): DeferredUpdates::runUpdate(MWCallableUpdate, Wikimedia\Rdbms\LBFactorySimple, integer)
#12 ...\mw-master\includes\deferred\DeferredUpdates.php(136): DeferredUpdates::execute(array, string, integer)
#13 ...\mw-master\includes\deferred\DeferredUpdates.php(95): DeferredUpdates::doUpdates(string)
#14 ...\mw-master\includes\deferred\DeferredUpdates.php(119): DeferredUpdates::addUpdate(MWCallableUpdate, integer)
#15 ...\mw-master\includes\libs\objectcache\BagOStuff.php(216): DeferredUpdates::addCallableUpdate(Closure)
#16 ...\mw-master\includes\libs\objectcache\BagOStuff.php(183): BagOStuff->trackDuplicateKeys(string)
#17 ...\mw-master\includes\jobqueue\JobQueue.php(519): BagOStuff->get(string)
#18 ...\mw-master\includes\jobqueue\JobQueue.php(501): JobQueue->doIsRootJobOldDuplicate(HTMLCacheUpdateJob)
#19 ...\mw-master\includes\jobqueue\JobQueue.php(381): JobQueue->isRootJobOldDuplicate(HTMLCacheUpdateJob)
#20 ...\mw-master\includes\jobqueue\JobQueueGroup.php(240): JobQueue->pop()
#21 ...\mw-master\includes\jobqueue\JobRunner.php(161): JobQueueGroup->pop(integer, integer, array)
#22 ...\mw-master\includes\MediaWiki.php(973): JobRunner->run(array)
#23 ...\mw-master\includes\MediaWiki.php(959): MediaWiki->triggerSyncJobs(integer, MediaWiki\Logger\LegacyLogger)
#24 ...\mw-master\includes\MediaWiki.php(911): MediaWiki->triggerJobs()
#25 ...\mw-master\includes\MediaWiki.php(731): MediaWiki->restInPeace(string)
#26 ...\mw-master\includes\MediaWiki.php(750): MediaWiki->{closure}()
#27 ...\mw-master\includes\MediaWiki.php(554): MediaWiki->doPostOutputShutdown(string)
#28 ...\mw-master\index.php(43): MediaWiki->run()
#29 {main}
`

Change 356120 had a related patch set uploaded (by Seb35; owner: Seb35):
[mediawiki/core@master] More properly close MediaWiki jobs

https://gerrit.wikimedia.org/r/356120

gerritbot added a project: Patch-For-Review.May 29 2017, 11:14 PM

I experienced this bug this Translate jobs, here is a proposed fix. I don’t really have the time now to explain more my findings, but I will take time later.

Probably not directly related to T157679, T154425, T154438 (I still experience them with my fix), although possibly similar/linked.

I experienced this bug this Translate jobs, here is a proposed fix. I don’t really have the time now to explain more my findings, but I will take time later.

Probably not directly related to T157679, T154425, T154438 (I still experience them with my fix), although possibly similar/linked.

Seb35 mentioned this in T154425: Delete action throws a DBUnexpectedError with "MWCallableUpdate::doUpdate: Cannot flush snapshot because writes are pending (JobQueueDB::doBatchPush)".May 30 2017, 8:23 AM

Seb35 mentioned this in T100085: PHP Notice: JobQueueGroup::__destruct: 1 buffered job(s) never inserted.May 31 2017, 9:09 AM

@demon This one also appears with 1.29!

• demon raised the priority of this task from Medium to Unbreak Now!.May 31 2017, 6:59 PM

• demon added a project: MW-1.29-release.

Restricted Application added subscribers: Jay8g, TerraCodes. · View Herald TranscriptMay 31 2017, 6:59 PM

Ciencia_Al_Poder assigned this task to Seb35.Jun 1 2017, 9:26 AM

Seb35 mentioned this in T154439: AutoCommitUpdate::doUpdate (Title->invalidateCache) causes Exception thrown with an uncommited database transaction.Jun 1 2017, 12:03 PM

Change 356120 merged by jenkins-bot:
[mediawiki/core@master] Better handling of jobs execution in post-connection shutdown

https://gerrit.wikimedia.org/r/356120

@mwjames Can you test with master version of MediaWiki? It should be fixed – Be warn: it does not fix (for now) the similar message from T154425.

Change 356799 had a related patch set uploaded (by Seb35; owner: Seb35):
[mediawiki/core@REL1_29] Better handling of jobs execution in post-connection shutdown

https://gerrit.wikimedia.org/r/356799

Cherry-picked to REL1_29. I guess it should be backported to REL1_27 and REL1_28, but I’m not sure. Will check later, except if someone is quicker.

Change 356799 merged by jenkins-bot:
[mediawiki/core@REL1_29] Better handling of jobs execution in post-connection shutdown

https://gerrit.wikimedia.org/r/356799

Change 356880 had a related patch set uploaded (by Seb35; owner: Seb35):
[mediawiki/core@REL1_28] Better handling of jobs execution in post-connection shutdown

https://gerrit.wikimedia.org/r/356880

Backported to 1.28, which is very similar to 1.29 in this area -- I was able to reproduce the same behaviour with Translate.

However it’s more different for 1.27 since it seems to work: Translate jobs are executed through the Web interface. Hence this task T165714 seems to be non-existant on 1.27 (because there is no mechanism DeferredUpdates::setImmediateMode) but the task T100085 is still experienced -- in this, it was not a so great idea to solve both bugs with the same gerrit patch.

Once patch for 1.28 is merged, this task can be closed.

Change 356880 merged by jenkins-bot:
[mediawiki/core@REL1_28] Better handling of jobs execution in post-connection shutdown

https://gerrit.wikimedia.org/r/356880

Can you test with master version of MediaWiki? It should be fixed

I did a quick test and it looks promising.

Seb35 updated the task description. (Show Details)Jun 3 2017, 10:50 AM

Seb35 mentioned this in T154438: Special:MovePage throws MWCallableUpdate::doUpdate: Cannot flush snapshot because writes are pending (JobQueueDB::doBatchPush, JobQueueDB::doBatchPush).

This should be fixed in all branches now. Resolve?

This should be fixed in all branches now. Resolve?

I could no longer find this particular error in logs, so yes this should be resolved.

Seb35 updated the task description. (Show Details)Jun 7 2017, 7:50 AM

Thanks for the confirmation @mwjames: given the possibly wide range of potential issues, it helps to have a +1 from a different setup.

I close this specific task given it was backported in all branches from 1.27 to 1.30 (thanks @demon and @aaron !) but the fix specifically for T100085 is modified by the fix of T154425, hence I let open some days these two other tasks during the backport to 1.27-1.29.