arlolra@tin:/srv/deployment/parsoid/deploy$ scap deploy-log -- Opening log file: '/srv/deployment/parsoid/deploy/scap/log/scap-sync-2017-09-14-0002-1-ga01064d8.log' 20:13:42 [tin] Started deploy [parsoid/deploy@a01064d] 20:13:42 [tin] Started deploy [parsoid/deploy@a01064d]: Updating Parsoid to 05a0965 20:13:42 [tin] == CANARY == :* wtp2001.codfw.wmnet :* wtp2002.codfw.wmnet :* wtp1002.eqiad.wmnet :* wtp1001.eqiad.wmnet 20:13:42 [wtp1002.eqiad.wmnet] Fetch from: http://tin.eqiad.wmnet/parsoid/deploy/.git 20:13:42 [wtp1001.eqiad.wmnet] Fetch from: http://tin.eqiad.wmnet/parsoid/deploy/.git 20:13:43 [wtp2002.codfw.wmnet] Fetch from: http://tin.eqiad.wmnet/parsoid/deploy/.git 20:13:43 [wtp2001.codfw.wmnet] Fetch from: http://tin.eqiad.wmnet/parsoid/deploy/.git 20:13:44 [wtp1002.eqiad.wmnet] Checkout rev: a01064d8ee0ae61caab9d1a24537a0867675b4b8 20:13:44 [wtp1001.eqiad.wmnet] Checkout rev: a01064d8ee0ae61caab9d1a24537a0867675b4b8 20:13:44 [wtp2001.codfw.wmnet] Checkout rev: a01064d8ee0ae61caab9d1a24537a0867675b4b8 20:13:44 [wtp2002.codfw.wmnet] Checkout rev: a01064d8ee0ae61caab9d1a24537a0867675b4b8 20:13:45 [wtp2001.codfw.wmnet] Sync submodules 20:13:45 [wtp2002.codfw.wmnet] Sync submodules 20:13:45 [wtp2001.codfw.wmnet] Update submodules 20:13:45 [wtp2001.codfw.wmnet] Updating .gitmodule: /srv/deployment/parsoid/deploy-cache/revs/a01064d8ee0ae61caab9d1a24537a0867675b4b8 20:13:46 [wtp2002.codfw.wmnet] Update submodules 20:13:46 [wtp2002.codfw.wmnet] Updating .gitmodule: /srv/deployment/parsoid/deploy-cache/revs/a01064d8ee0ae61caab9d1a24537a0867675b4b8 20:13:46 [wtp1002.eqiad.wmnet] Sync submodules 20:13:47 [wtp1002.eqiad.wmnet] Update submodules 20:13:47 [wtp1002.eqiad.wmnet] Updating .gitmodule: /srv/deployment/parsoid/deploy-cache/revs/a01064d8ee0ae61caab9d1a24537a0867675b4b8 20:13:47 [wtp1001.eqiad.wmnet] Sync submodules 20:13:47 [wtp1001.eqiad.wmnet] Update submodules 20:13:47 [wtp1001.eqiad.wmnet] Updating .gitmodule: /srv/deployment/parsoid/deploy-cache/revs/a01064d8ee0ae61caab9d1a24537a0867675b4b8 20:14:13 [wtp1001.eqiad.wmnet] Rendering config_file: /srv/deployment/parsoid/deploy-cache/revs/a01064d8ee0ae61caab9d1a24537a0867675b4b8/.git/config-files/etc/parsoid/config.yaml 20:14:13 [wtp2002.codfw.wmnet] Rendering config_file: /srv/deployment/parsoid/deploy-cache/revs/a01064d8ee0ae61caab9d1a24537a0867675b4b8/.git/config-files/etc/parsoid/config.yaml 20:14:13 [wtp2001.codfw.wmnet] Rendering config_file: /srv/deployment/parsoid/deploy-cache/revs/a01064d8ee0ae61caab9d1a24537a0867675b4b8/.git/config-files/etc/parsoid/config.yaml 20:14:13 [wtp1002.eqiad.wmnet] Rendering config_file: /srv/deployment/parsoid/deploy-cache/revs/a01064d8ee0ae61caab9d1a24537a0867675b4b8/.git/config-files/etc/parsoid/config.yaml 20:14:14 [wtp1001.eqiad.wmnet] Linking config files at: /srv/deployment/parsoid/deploy-cache/revs/a01064d8ee0ae61caab9d1a24537a0867675b4b8/.git/config-files 20:14:14 [wtp1001.eqiad.wmnet] Executing check 'depool' 20:14:14 [wtp1002.eqiad.wmnet] Linking config files at: /srv/deployment/parsoid/deploy-cache/revs/a01064d8ee0ae61caab9d1a24537a0867675b4b8/.git/config-files 20:14:14 [wtp1002.eqiad.wmnet] Executing check 'depool' 20:14:14 [wtp2001.codfw.wmnet] Linking config files at: /srv/deployment/parsoid/deploy-cache/revs/a01064d8ee0ae61caab9d1a24537a0867675b4b8/.git/config-files 20:14:14 [wtp2001.codfw.wmnet] Executing check 'depool' 20:14:14 [wtp2002.codfw.wmnet] Linking config files at: /srv/deployment/parsoid/deploy-cache/revs/a01064d8ee0ae61caab9d1a24537a0867675b4b8/.git/config-files 20:14:14 [wtp2002.codfw.wmnet] Executing check 'depool' 20:14:18 [wtp1002.eqiad.wmnet] Check 'depool' failed: WARNING:etcd.client:etcd response did not contain a cluster ID ERROR:conftool:Error when trying to set/pooled=no on service=parsoid,name=wtp1002.eqiad.wmnet ERROR:conftool:Failure writing to the kvstore: Backend error: The request requires user authentication : Insufficient credentials WARNING:etcd.client:etcd response did not contain a cluster ID ERROR:conftool:Error when trying to set/pooled=no on service=parsoid,name=wtp1002.eqiad.wmnet ERROR:conftool:Failure writing to the kvstore: Backend error: The request requires user authentication : Insufficient credentials WARNING:etcd.client:etcd response did not contain a cluster ID ERROR:conftool:Error when trying to set/pooled=no on service=parsoid,name=wtp1002.eqiad.wmnet ERROR:conftool:Failure writing to the kvstore: Backend error: The request requires user authentication : Insufficient credentials 20:14:18 [tin] [u'/usr/bin/scap', u'deploy-local', u'-v', u'--repo', u'parsoid/deploy', u'-g', u'canary', u'promote', u'--refresh-config'] on wtp1002.eqiad.wmnet returned [1]: 20:14:18 [wtp1001.eqiad.wmnet] Check 'depool' failed: WARNING:etcd.client:etcd response did not contain a cluster ID ERROR:conftool:Error when trying to set/pooled=no on service=parsoid,name=wtp1001.eqiad.wmnet ERROR:conftool:Failure writing to the kvstore: Backend error: The request requires user authentication : Insufficient credentials WARNING:etcd.client:etcd response did not contain a cluster ID ERROR:conftool:Error when trying to set/pooled=no on service=parsoid,name=wtp1001.eqiad.wmnet ERROR:conftool:Failure writing to the kvstore: Backend error: The request requires user authentication : Insufficient credentials WARNING:etcd.client:etcd response did not contain a cluster ID ERROR:conftool:Error when trying to set/pooled=no on service=parsoid,name=wtp1001.eqiad.wmnet ERROR:conftool:Failure writing to the kvstore: Backend error: The request requires user authentication : Insufficient credentials 20:14:18 [tin] [u'/usr/bin/scap', u'deploy-local', u'-v', u'--repo', u'parsoid/deploy', u'-g', u'canary', u'promote', u'--refresh-config'] on wtp1001.eqiad.wmnet returned [1]: 20:14:20 [wtp2001.codfw.wmnet] Check 'depool' failed: WARNING:etcd.client:etcd response did not contain a cluster ID ERROR:conftool:Error when trying to set/pooled=no on service=parsoid,name=wtp2001.codfw.wmnet ERROR:conftool:Failure writing to the kvstore: Backend error: The request requires user authentication : Insufficient credentials WARNING:etcd.client:etcd response did not contain a cluster ID ERROR:conftool:Error when trying to set/pooled=no on service=parsoid,name=wtp2001.codfw.wmnet ERROR:conftool:Failure writing to the kvstore: Backend error: The request requires user authentication : Insufficient credentials WARNING:etcd.client:etcd response did not contain a cluster ID ERROR:conftool:Error when trying to set/pooled=no on service=parsoid,name=wtp2001.codfw.wmnet ERROR:conftool:Failure writing to the kvstore: Backend error: The request requires user authentication : Insufficient credentials 20:14:20 [tin] [u'/usr/bin/scap', u'deploy-local', u'-v', u'--repo', u'parsoid/deploy', u'-g', u'canary', u'promote', u'--refresh-config'] on wtp2001.codfw.wmnet returned [1]: 20:14:20 [wtp2002.codfw.wmnet] Check 'depool' failed: WARNING:etcd.client:etcd response did not contain a cluster ID ERROR:conftool:Error when trying to set/pooled=no on service=parsoid,name=wtp2002.codfw.wmnet ERROR:conftool:Failure writing to the kvstore: Backend error: The request requires user authentication : Insufficient credentials WARNING:etcd.client:etcd response did not contain a cluster ID ERROR:conftool:Error when trying to set/pooled=no on service=parsoid,name=wtp2002.codfw.wmnet ERROR:conftool:Failure writing to the kvstore: Backend error: The request requires user authentication : Insufficient credentials WARNING:etcd.client:etcd response did not contain a cluster ID ERROR:conftool:Error when trying to set/pooled=no on service=parsoid,name=wtp2002.codfw.wmnet ERROR:conftool:Failure writing to the kvstore: Backend error: The request requires user authentication : Insufficient credentials 20:14:20 [tin] [u'/usr/bin/scap', u'deploy-local', u'-v', u'--repo', u'parsoid/deploy', u'-g', u'canary', u'promote', u'--refresh-config'] on wtp2002.codfw.wmnet returned [1]: 20:14:20 [tin] 4 targets had deploy errors 20:14:20 [tin] 4 targets failed 20:14:20 [tin] 4 of 4 canary targets failed, exceeding limit 20:15:16 [tin] == CANARY == :* wtp2001.codfw.wmnet :* wtp2002.codfw.wmnet :* wtp1002.eqiad.wmnet :* wtp1001.eqiad.wmnet 20:15:16 [wtp1001.eqiad.wmnet] Rolling back from revision a01064d8ee0ae61caab9d1a24537a0867675b4b8 to cec7d1762e2cf09c61279ed66e6520e1f4ea0744 20:15:16 [wtp1001.eqiad.wmnet] Removing old revision /srv/deployment/parsoid/deploy-cache/revs/a9dc803de9e599c8620b2940a4f4a152d98bcdb4 20:15:16 [wtp1002.eqiad.wmnet] Rolling back from revision a01064d8ee0ae61caab9d1a24537a0867675b4b8 to cec7d1762e2cf09c61279ed66e6520e1f4ea0744 20:15:16 [wtp1002.eqiad.wmnet] Removing old revision /srv/deployment/parsoid/deploy-cache/revs/a9dc803de9e599c8620b2940a4f4a152d98bcdb4 20:15:17 [wtp2002.codfw.wmnet] Rolling back from revision a01064d8ee0ae61caab9d1a24537a0867675b4b8 to cec7d1762e2cf09c61279ed66e6520e1f4ea0744 20:15:17 [wtp2001.codfw.wmnet] Rolling back from revision a01064d8ee0ae61caab9d1a24537a0867675b4b8 to cec7d1762e2cf09c61279ed66e6520e1f4ea0744 20:15:17 [wtp2001.codfw.wmnet] Removing old revision /srv/deployment/parsoid/deploy-cache/revs/a9dc803de9e599c8620b2940a4f4a152d98bcdb4 20:15:17 [wtp2002.codfw.wmnet] Removing old revision /srv/deployment/parsoid/deploy-cache/revs/a9dc803de9e599c8620b2940a4f4a152d98bcdb4 20:15:17 [wtp1001.eqiad.wmnet] Restarting service 'parsoid' 20:15:17 [wtp1002.eqiad.wmnet] Restarting service 'parsoid' 20:15:17 [wtp2002.codfw.wmnet] Restarting service 'parsoid' 20:15:17 [wtp2001.codfw.wmnet] Restarting service 'parsoid' 20:15:21 [wtp1001.eqiad.wmnet] Executing check 'repool' 20:15:21 [wtp1002.eqiad.wmnet] Executing check 'repool' 20:15:21 [wtp2002.codfw.wmnet] Executing check 'repool' 20:15:25 [wtp1001.eqiad.wmnet] Check 'repool' failed: WARNING:etcd.client:etcd response did not contain a cluster ID ERROR:conftool:Error when trying to set/pooled=yes on service=parsoid,name=wtp1001.eqiad.wmnet ERROR:conftool:Failure writing to the kvstore: Backend error: The request requires user authentication : Insufficient credentials WARNING:etcd.client:etcd response did not contain a cluster ID ERROR:conftool:Error when trying to set/pooled=yes on service=parsoid,name=wtp1001.eqiad.wmnet ERROR:conftool:Failure writing to the kvstore: Backend error: The request requires user authentication : Insufficient credentials WARNING:etcd.client:etcd response did not contain a cluster ID ERROR:conftool:Error when trying to set/pooled=yes on service=parsoid,name=wtp1001.eqiad.wmnet ERROR:conftool:Failure writing to the kvstore: Backend error: The request requires user authentication : Insufficient credentials 20:15:25 [tin] [u'/usr/bin/scap', u'deploy-local', u'-v', u'--repo', u'parsoid/deploy', u'-g', u'canary', u'rollback', u'--refresh-config'] on wtp1001.eqiad.wmnet returned [1]: 20:15:26 [wtp1002.eqiad.wmnet] Check 'repool' failed: WARNING:etcd.client:etcd response did not contain a cluster ID ERROR:conftool:Error when trying to set/pooled=yes on service=parsoid,name=wtp1002.eqiad.wmnet ERROR:conftool:Failure writing to the kvstore: Backend error: The request requires user authentication : Insufficient credentials WARNING:etcd.client:etcd response did not contain a cluster ID ERROR:conftool:Error when trying to set/pooled=yes on service=parsoid,name=wtp1002.eqiad.wmnet ERROR:conftool:Failure writing to the kvstore: Backend error: The request requires user authentication : Insufficient credentials WARNING:etcd.client:etcd response did not contain a cluster ID ERROR:conftool:Error when trying to set/pooled=yes on service=parsoid,name=wtp1002.eqiad.wmnet ERROR:conftool:Failure writing to the kvstore: Backend error: The request requires user authentication : Insufficient credentials 20:15:26 [tin] [u'/usr/bin/scap', u'deploy-local', u'-v', u'--repo', u'parsoid/deploy', u'-g', u'canary', u'rollback', u'--refresh-config'] on wtp1002.eqiad.wmnet returned [1]: 20:15:27 [wtp2002.codfw.wmnet] Check 'repool' failed: WARNING:etcd.client:etcd response did not contain a cluster ID ERROR:conftool:Error when trying to set/pooled=yes on service=parsoid,name=wtp2002.codfw.wmnet ERROR:conftool:Failure writing to the kvstore: Backend error: The request requires user authentication : Insufficient credentials WARNING:etcd.client:etcd response did not contain a cluster ID ERROR:conftool:Error when trying to set/pooled=yes on service=parsoid,name=wtp2002.codfw.wmnet ERROR:conftool:Failure writing to the kvstore: Backend error: The request requires user authentication : Insufficient credentials WARNING:etcd.client:etcd response did not contain a cluster ID ERROR:conftool:Error when trying to set/pooled=yes on service=parsoid,name=wtp2002.codfw.wmnet ERROR:conftool:Failure writing to the kvstore: Backend error: The request requires user authentication : Insufficient credentials 20:15:27 [tin] [u'/usr/bin/scap', u'deploy-local', u'-v', u'--repo', u'parsoid/deploy', u'-g', u'canary', u'rollback', u'--refresh-config'] on wtp2002.codfw.wmnet returned [1]: 20:15:30 [wtp2001.codfw.wmnet] Executing check 'repool' 20:15:36 [wtp2001.codfw.wmnet] Check 'repool' failed: WARNING:etcd.client:etcd response did not contain a cluster ID ERROR:conftool:Error when trying to set/pooled=yes on service=parsoid,name=wtp2001.codfw.wmnet ERROR:conftool:Failure writing to the kvstore: Backend error: The request requires user authentication : Insufficient credentials WARNING:etcd.client:etcd response did not contain a cluster ID ERROR:conftool:Error when trying to set/pooled=yes on service=parsoid,name=wtp2001.codfw.wmnet ERROR:conftool:Failure writing to the kvstore: Backend error: The request requires user authentication : Insufficient credentials WARNING:etcd.client:etcd response did not contain a cluster ID ERROR:conftool:Error when trying to set/pooled=yes on service=parsoid,name=wtp2001.codfw.wmnet ERROR:conftool:Failure writing to the kvstore: Backend error: The request requires user authentication : Insufficient credentials 20:15:36 [tin] [u'/usr/bin/scap', u'deploy-local', u'-v', u'--repo', u'parsoid/deploy', u'-g', u'canary', u'rollback', u'--refresh-config'] on wtp2001.codfw.wmnet returned [1]: 20:15:36 [tin] 4 targets had deploy errors 20:15:36 [tin] 4 targets failed 20:15:36 [tin] Finished deploy [parsoid/deploy@a01064d]: Updating Parsoid to 05a0965 (duration: 01m 53s) 20:15:36 [tin] Finished deploy [parsoid/deploy@a01064d] (duration: 01m 53s)