beta cluster scap failure
Closed, ResolvedPublic
Actions

Description

https://integration.wikimedia.org/ci/job/beta-scap-eqiad/50109/console

21:03:58 21:03:58 Started scap: beta-scap-eqiad (build #50109)
21:03:58 21:03:58 Copying to deployment-bastion.eqiad.wmflabs from deployment-bastion.eqiad.wmflabs
21:03:58 21:03:58 Started rsync common
21:04:07 21:04:07 Finished rsync common (duration: 00m 08s)
21:04:07 21:04:07 Compiled /srv/mediawiki/wikiversions-labs.json to /srv/mediawiki/wikiversions-labs.cdb
21:04:07 21:04:07 Started mw-update-l10n
21:04:07 21:04:07 Updating ExtensionMessages-master.php
21:04:08 21:04:08 Finished mw-update-l10n (duration: 00m 00s)
21:04:08 21:04:08 Unhandled error:
21:04:08 Traceback (most recent call last):
21:04:08   File "/mnt/srv/deployment/scap/scap/scap/cli.py", line 276, in run
21:04:08     exit_status = app.main(extra_args)
21:04:08   File "/mnt/srv/deployment/scap/scap/scap/main.py", line 39, in main
21:04:08     self._before_cluster_sync()
21:04:08   File "/mnt/srv/deployment/scap/scap/scap/main.py", line 218, in _before_cluster_sync
21:04:08     version, wikidb, self.verbose, self.config)
21:04:08   File "/mnt/srv/deployment/scap/scap/scap/tasks.py", line 496, in update_localization_cache
21:04:08     logger)
21:04:08   File "/mnt/srv/deployment/scap/scap/scap/utils.py", line 308, in sudo_check_call
21:04:08     raise subprocess.CalledProcessError(proc.returncode, cmd)
21:04:08 CalledProcessError: Command '/usr/local/bin/mwscript mergeMessageFileList.php --wiki="eowiki" --list-file="/srv/mediawiki-staging/wmf-config/extension-list" --output="/tmp/tmp.VpRueQKWIR" ' returned non-zero exit status 1

Related Objects
Search...

Status	Assigned	Task
Resolved	zeljkofilipin	T94150 Fix scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily
Resolved	Jdlrobson	T94156 Fix MobileFrontend scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily
Resolved	thcipriani	T96920 beta cluster scap failure
Resolved	bd808	T97138 Beta cluster: exception 'LogicException' with message 'Missing stream uri, the stream can not be opened.' in /mnt/srv/mediawiki-staging/php-master/includes/debug/logger/monolog/LegacyHandler.php:113

Event Timeline

greg created this task.Apr 22 2015, 9:11 PM

greg assigned this task to mmodell.

greg raised the priority of this task from to Unbreak Now!.

greg updated the task description. (Show Details)

greg added a project: Beta-Cluster-Infrastructure.

greg added subscribers: greg, thcipriani, dduvall, Jdlrobson.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 22 2015, 9:11 PM

Jdlrobson set Security to None.Apr 22 2015, 10:07 PM

Jdlrobson added a subscriber: JKatzWMF.

Jdlrobson added a parent task: T94156: Fix MobileFrontend scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily.Apr 23 2015, 7:02 PM

Jdlrobson mentioned this in T94156: Fix MobileFrontend scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily.

@mmodell @thcipriani ping (this looks unrelated to the flappiness of wmf labs as it started before all of that)

It seems scap strip the underlying command output. Running mwscript manually I got:

Fatal error: Uncaught exception 'LogicException' with message 'Missing stream uri, the stream can not be opened.' in /mnt/srv/mediawiki-staging/php-master/includes/debug/logger/monolog/LegacyHandler.php:113
Stack trace:
#0 /mnt/srv/mediawiki-staging/php-master/includes/debug/logger/monolog/LegacyHandler.php(197): MediaWiki\Logger\Monolog\LegacyHandler->openSink()
#1 /mnt/srv/mediawiki-staging/php-master/vendor/monolog/monolog/src/Monolog/Handler/AbstractProcessingHandler.php(37): MediaWiki\Logger\Monolog\LegacyHandler->write(Array)
#2 /mnt/srv/mediawiki-staging/php-master/vendor/monolog/monolog/src/Monolog/Logger.php(265): Monolog\Handler\AbstractProcessingHandler->handle(Array)
#3 /mnt/srv/mediawiki-staging/php-master/vendor/monolog/monolog/src/Monolog/Logger.php(473): Monolog\Logger->addRecord(200, '[163d8839] [no ...', Array)
#4 /mnt/srv/mediawiki-staging/php-master/includes/GlobalFunctions.php(1157): Monolog\Logger->info('[163d8839] [no ...', Array)
#5 /mnt/srv/mediawiki-staging/php-master/includes/exception/MWExcepti in /mnt/srv/mediawiki-staging/php-master/includes/debug/logger/monolog/LegacyHandler.php on line 113

Which might be related to the code being outdated or some recent logging config change. Looking at the database update job, only testwiki seems to be able to run! And /usr/local/bin/mwscript eval.php --wiki="testwiki" works.

Filled as T97138

hashar added a subtask: T97138: Beta cluster: exception 'LogicException' with message 'Missing stream uri, the stream can not be opened.' in /mnt/srv/mediawiki-staging/php-master/includes/debug/logger/monolog/LegacyHandler.php:113.Apr 24 2015, 2:51 PM