Page MenuHomePhabricator

JobQueueError: Could not enqueue jobs from stream [stream]
Closed, DuplicatePublicPRODUCTION ERROR

Description

Error

MediaWiki version: 1.36.0-wmf.4

message
Could not enqueue jobs from stream mediawiki.job.wikibase-addUsagesForPage

Impact/Notes

  • Shows up on a handful of wikis for a handful of different files
  • No additional errors in stack trace or message gives clue about the root cause
  • Several different streams show up mediawiki.job.wikibase-addUsagesForPage, mediawiki.job.cirrusSearchElasticaWrite

Details

Request ID
8fc78714-6e08-4a4e-b747-55ff300c75d1
Request URL
https://commons.wikimedia.org/w/index.php?title=Image:Flag_of_the_Northwest_Territories.svg&action=render&uselang=hr
Stack Trace
exception.trace
#0 /srv/mediawiki/php-1.36.0-wmf.4/includes/jobqueue/JobQueue.php(374): MediaWiki\Extension\EventBus\Adapters\JobQueue\JobQueueEventBus->doBatchPush(array, integer)
#1 /srv/mediawiki/php-1.36.0-wmf.4/includes/jobqueue/JobQueue.php(344): JobQueue->batchPush(array, integer)
#2 /srv/mediawiki/php-1.36.0-wmf.4/includes/jobqueue/JobQueueGroup.php(171): JobQueue->push(array)
#3 /srv/mediawiki/php-1.36.0-wmf.4/extensions/Wikibase/client/includes/Hooks/DataUpdateHookHandler.php(174): JobQueueGroup->push(array)
#4 /srv/mediawiki/php-1.36.0-wmf.4/includes/deferred/MWCallableUpdate.php(38): Wikibase\Client\Hooks\DataUpdateHookHandler->Wikibase\Client\Hooks\{closure}()
#5 /srv/mediawiki/php-1.36.0-wmf.4/includes/deferred/DeferredUpdates.php(467): MWCallableUpdate->doUpdate()
#6 /srv/mediawiki/php-1.36.0-wmf.4/includes/deferred/DeferredUpdates.php(344): DeferredUpdates::attemptUpdate(MWCallableUpdate, Wikimedia\Rdbms\LBFactoryMulti)
#7 /srv/mediawiki/php-1.36.0-wmf.4/includes/deferred/DeferredUpdates.php(278): DeferredUpdates::run(MWCallableUpdate, Wikimedia\Rdbms\LBFactoryMulti, Monolog\Logger, BufferingStatsdDataFactory, string)
#8 /srv/mediawiki/php-1.36.0-wmf.4/includes/deferred/DeferredUpdates.php(194): DeferredUpdates::handleUpdateQueue(array, string, integer)
#9 /srv/mediawiki/php-1.36.0-wmf.4/includes/MediaWiki.php(1113): DeferredUpdates::doUpdates(string)
#10 /srv/mediawiki/php-1.36.0-wmf.4/includes/MediaWiki.php(849): MediaWiki->restInPeace()
#11 /srv/mediawiki/php-1.36.0-wmf.4/includes/MediaWiki.php(861): MediaWiki->{closure}()
#12 /srv/mediawiki/php-1.36.0-wmf.4/includes/MediaWiki.php(582): MediaWiki->doPostOutputShutdown()
#13 /srv/mediawiki/php-1.36.0-wmf.4/index.php(53): MediaWiki->run()
#14 /srv/mediawiki/php-1.36.0-wmf.4/index.php(46): wfIndexMain()
#15 /srv/mediawiki/w/index.php(3): require(string)
#16 {main}

Event Timeline

thcipriani renamed this task from JobQueueError: Could not enqueue jobs from stream mediawiki.job.wikibase-addUsagesForPage to JobQueueError: Could not enqueue jobs from stream [stream].Aug 12 2020, 6:36 PM
thcipriani updated the task description. (Show Details)

Searching for the request-id unveils that the root cause is Unable to deliver all events: 503: Service Unavailable for EventBus->eventgate request.

Thus I think the root cause is T249745 which has been happening for awhile. @Clarakosi touched some code around it in https://gerrit.wikimedia.org/r/c/mediawiki/extensions/EventBus/+/618353

Change 620060 had a related patch set uploaded (by Clarakosi; owner: Clarakosi):
[mediawiki/extensions/EventBus@master] JobQueue: Improve error logging

https://gerrit.wikimedia.org/r/620060

Change 620060 abandoned by Clarakosi:
[mediawiki/extensions/EventBus@master] JobQueue: Improve error logging

Reason:
Error is already being logged to the EventBus channel

https://gerrit.wikimedia.org/r/620060

This is still occurring as of Sept 23, 2020.