2018-02-05 MCS Beta Cluster deployment failed with error
Closed, ResolvedPublic


Deployment worked without issue in production, but hit the following error when deploying to the Beta Cluster:

15:57:50 [deployment-mcs01.deployment-prep.eqiad.wmflabs] <Command u'/usr/bin/git submodule update --init --recursive --jobs 1 --reference /srv/deployment/mobileapps/deploy-cache/cache'>: starting process
15:57:50 [deployment-mcs01.deployment-prep.eqiad.wmflabs] <Command u'/usr/bin/git submodule update --init --recursive --jobs 1 --reference /srv/deployment/mobileapps/deploy-cache/cache', pid 6739>: process started
15:57:52 [deployment-mcs01.deployment-prep.eqiad.wmflabs] <Command u'/usr/bin/git submodule update --init --recursive --jobs 1 --reference /srv/deployment/mobileapps/deploy-cache/cache', pid 6739>: process completed
15:57:52 [deployment-mcs01.deployment-prep.eqiad.wmflabs] Unhandled error:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/scap/cli.py", line 330, in run
    exit_status = app.main(app.extra_arguments)
  File "/usr/lib/python2.7/dist-packages/scap/deploy.py", line 146, in main
    getattr(self, stage)()
  File "/usr/lib/python2.7/dist-packages/scap/deploy.py", line 332, in fetch
    for manager in git_binary_manager:
TypeError: 'NoneType' object is not iterable
15:57:52 [deployment-mcs01.deployment-prep.eqiad.wmflabs] deploy-local failed: <TypeError> {}
15:57:52 [deployment-tin] [u'/usr/bin/scap', u'deploy-local', u'-v', u'--repo', u'mobileapps/deploy', u'-g', u'default', u'fetch', u'--refresh-config'] on deployment-mcs01.deployment-prep.eqiad.wmflabs returned [70]: http://deployment-tin.deployment-prep.eqiad.wmflabs/mobileapps/deploy/.git

15:57:52 [deployment-tin] 1 targets had deploy errors
15:57:52 [deployment-tin] 1 targets failed
15:57:52 [deployment-tin] 1 of 1 default targets failed, exceeding limit
Mholloway created this task.Feb 5 2018, 4:07 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Mholloway updated the task description. (Show Details)Feb 5 2018, 4:08 PM

Mentioned in SAL (#wikimedia-releng) [2018-02-05T16:09:05Z] <mdholloway> mobileapps deployment to BC failed with error (T186532)

thcipriani triaged this task as High priority.Feb 5 2018, 4:26 PM
thcipriani added a subscriber: thcipriani.

Seems like this is a problem with D921

If git_binary_manager is set to None (the default), then this errors out.

thcipriani closed this task as Resolved.
thcipriani claimed this task.

This should be resolved once scap 3.8.0-1~20180205205900.289 is installed on all targets via puppet (should happen within the next 30 minutes or so. Tested working from deployment-tin -> deployment-mediawiki05.

Confirming it works well for me again (deployed to deployment-mcs01).