Page MenuHomePhabricator
Paste P30044

(An Untitled Masterwork)
ActivePublic

Authored by Dzahn on Jun 23 2022, 8:54 PM.
Tags
None
Referenced Files
F35269174: raw-paste-data.txt
Jun 23 2022, 8:54 PM
Subscribers
None
================
PASS | | 0% (0/14) [00:23<?, ?hosts/s]
FAIL |███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100% (14/14) [00:23<00:00, 1.68s/hosts]
100.0% (14/14) of nodes failed to execute command 'cd /srv/authdns/...nippets --deploy': authdns[1001,2001].wikimedia.org,dns[1001-1002,2001-2002,3001-3002,4001-4002,5001-5002,6001-6002].wikimedia.org
0.0% (0/14) success ratio (< 100.0% threshold) for command: 'cd /srv/authdns/...nippets --deploy'. Aborting.
0.0% (0/14) success ratio (< 100.0% threshold) of nodes successfully executed all commands. Aborting.
Failed to run spicerack.remote.RemoteHosts.run_sync: Cumin execution failed (exit_code=2)
==> What do you want to do? "retry" the last command, manually fix the issue and "skip" the last command to continue the execution or completely "abort" the execution.
> abort
Exception raised while executing cookbook sre.dns.netbox:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/wmflib/interactive.py", line 145, in confirm_on_failure
ret = func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/spicerack/remote.py", line 520, in run_sync
return self._execute(
File "/usr/lib/python3/dist-packages/spicerack/remote.py", line 720, in _execute
raise RemoteExecutionError(ret, "Cumin execution failed")
spicerack.remote.RemoteExecutionError: Cumin execution failed (exit_code=2)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/spicerack/_menu.py", line 234, in run
raw_ret = runner.run()
File "/usr/lib/python3/dist-packages/spicerack/_module_api.py", line 18, in run
return self._run(self.args, self.spicerack)
File "/srv/deployment/spicerack/cookbooks/sre/dns/netbox.py", line 128, in run
confirm_on_failure(
File "/usr/lib/python3/dist-packages/wmflib/interactive.py", line 155, in confirm_on_failure
raise AbortError('Task manually aborted') from e
wmflib.interactive.AbortError: Task manually aborted
END (FAIL) - Cookbook sre.dns.netbox (exit_code=99)
START - Cookbook sre.dns.wipe-cache dse-k8s-ctrl1001.eqiad.wmnet on all recursors