Page MenuHomePhabricator

JobQueueEventBus->doBatchPush times out while trying to send Echo notifications about a Flow edit
Closed, DeclinedPublicPRODUCTION ERROR

Description

Error
normalized_message
Deferred update 'MWCallableUpdate_Flow\SubmissionHandler->commit' failed to run.
exception.trace
from /srv/mediawiki/php-1.37.0-wmf.21/vendor/wikimedia/request-timeout/src/Detail/ExcimerTimerWrapper.php(97)
#0 /srv/mediawiki/php-1.37.0-wmf.21/vendor/wikimedia/request-timeout/src/Detail/ExcimerTimerWrapper.php(72): Wikimedia\RequestTimeout\Detail\ExcimerTimerWrapper->onTimeout(integer)
#1 /srv/mediawiki/php-1.37.0-wmf.21/includes/libs/http/MultiHttpClient.php(248): Wikimedia\RequestTimeout\Detail\ExcimerTimerWrapper->Wikimedia\RequestTimeout\Detail\{closure}(integer)
#2 /srv/mediawiki/php-1.37.0-wmf.21/includes/libs/http/MultiHttpClient.php(189): MultiHttpClient->runMultiCurl(array, array)
#3 /srv/mediawiki/php-1.37.0-wmf.21/extensions/EventBus/includes/EventBus.php(244): MultiHttpClient->runMulti(array, array)
#4 /srv/mediawiki/php-1.37.0-wmf.21/extensions/EventBus/includes/Adapters/JobQueue/JobQueueEventBus.php(112): MediaWiki\Extension\EventBus\EventBus->send(array, integer)
#5 /srv/mediawiki/php-1.37.0-wmf.21/includes/jobqueue/JobQueue.php(364): MediaWiki\Extension\EventBus\Adapters\JobQueue\JobQueueEventBus->doBatchPush(array, integer)
#6 /srv/mediawiki/php-1.37.0-wmf.21/includes/jobqueue/JobQueue.php(336): JobQueue->batchPush(array, integer)
#7 /srv/mediawiki/php-1.37.0-wmf.21/includes/jobqueue/JobQueueGroup.php(182): JobQueue->push(array)
#8 /srv/mediawiki/php-1.37.0-wmf.21/extensions/Echo/includes/Push/PushNotifier.php(22): JobQueueGroup->push(array)
#9 /srv/mediawiki/php-1.37.0-wmf.21/extensions/Echo/includes/controller/NotificationController.php(423): EchoPush\PushNotifier::notifyWithPush(User, EchoEvent)
#10 /srv/mediawiki/php-1.37.0-wmf.21/extensions/Echo/includes/controller/NotificationController.php(131): EchoNotificationController::doNotification(EchoEvent, User, string)
#11 /srv/mediawiki/php-1.37.0-wmf.21/extensions/Echo/includes/model/Event.php(195): EchoNotificationController::notify(EchoEvent, boolean)
#12 /srv/mediawiki/php-1.37.0-wmf.21/extensions/Flow/includes/Notifications/Controller.php(427): EchoEvent::create(array)
#13 /srv/mediawiki/php-1.37.0-wmf.21/extensions/Flow/includes/Data/Listener/NotificationListener.php(48): Flow\Notifications\Controller->notifyNewTopic(array)
#14 /srv/mediawiki/php-1.37.0-wmf.21/extensions/Flow/includes/Data/Listener/DeferredInsertLifecycleHandler.php(36): Flow\Data\Listener\NotificationListener->onAfterInsert(Flow\Model\PostRevision, array, array)
#15 /srv/mediawiki/php-1.37.0-wmf.21/includes/deferred/MWCallableUpdate.php(38): Flow\Data\Listener\DeferredInsertLifecycleHandler::Flow\Data\Listener\{closure}()
#16 /srv/mediawiki/php-1.37.0-wmf.21/includes/deferred/DeferredUpdates.php(515): MWCallableUpdate->doUpdate()
#17 /srv/mediawiki/php-1.37.0-wmf.21/includes/deferred/DeferredUpdates.php(391): DeferredUpdates::attemptUpdate(MWCallableUpdate, Wikimedia\Rdbms\LBFactoryMulti)
#18 /srv/mediawiki/php-1.37.0-wmf.21/includes/deferred/DeferredUpdates.php(221): DeferredUpdates::run(MWCallableUpdate, Wikimedia\Rdbms\LBFactoryMulti, Monolog\Logger, BufferingStatsdDataFactory, string)
#19 /srv/mediawiki/php-1.37.0-wmf.21/includes/deferred/DeferredUpdatesScope.php(267): DeferredUpdates::{closure}(MWCallableUpdate, integer)
#20 /srv/mediawiki/php-1.37.0-wmf.21/includes/deferred/DeferredUpdatesScope.php(196): DeferredUpdatesScope->processStageQueue(integer, integer, Closure)
#21 /srv/mediawiki/php-1.37.0-wmf.21/includes/deferred/DeferredUpdates.php(242): DeferredUpdatesScope->processUpdates(integer, Closure)
#22 /srv/mediawiki/php-1.37.0-wmf.21/includes/MediaWiki.php(1136): DeferredUpdates::doUpdates(string)
#23 /srv/mediawiki/php-1.37.0-wmf.21/includes/MediaWiki.php(846): MediaWiki->restInPeace()
#24 /srv/mediawiki/php-1.37.0-wmf.21/includes/MediaWiki.php(584): MediaWiki->doPostOutputShutdown()
#25 /srv/mediawiki/php-1.37.0-wmf.21/index.php(53): MediaWiki->run()
#26 /srv/mediawiki/php-1.37.0-wmf.21/index.php(46): wfIndexMain()
#27 /srv/mediawiki/w/index.php(3): require(string)
#28 {main}
Impact
Notes

Details

Request URL
https://fr.wikipedia.org/w/index.php?title=*&action=*

Event Timeline

Moving to Needs Discussion to determine impact

@mewoph Thanks! Who needs to be involved in discussing impact?

Tgr subscribed.

This is a job queue infrastructure failure based on the stack trace (EventBus tries to send a job queue push event and times out). Not sure if the PET or ServiceOps teams want to do something about it, for us I don't think it's actionable.

Tgr renamed this task from Deferred update 'MWCallableUpdate_Flow\SubmissionHandler->commit' failed to run. to JobQueueEventBus->doBatchPush times out while trying to send Echo notifications about a Flow edit.Sep 8 2021, 2:52 AM

I'll tag as serviceops for awareness, feel free to remove if it's not in scope.

Not seen in the past 30 days, except for one instance with a different underlying cause:

Aug 1, 2023 @ 23:22:16.425DeferredUpdatesmw1400frwikiDeferred update 'MWCallableUpdate_Flow\SubmissionHandler->commit' failed to run.
  • exception.previous: InvalidArgumentException: Cannot notify anonymous user: Unknown user
  • url: /w/api.php