Page MenuHomePhabricator
Paste P21507

homer failure during decom of elastic10[39|43].eqiad.wmnet
ActivePublic
Actions

Authored by RKemper on Feb 24 2022, 6:07 PM.
Running Homer on asw2-c-eqiad.mgmt.eqiad.wmnet, it takes time ⏳ don't worry
INFO:homer.devices:Initialized 53 devices
INFO:homer:Committing config for query asw2-c-eqiad.mgmt.eqiad.wmnet with message: Host decommission - ryankemper@cumin1001 - T294805
INFO:homer:Gathering global Netbox data
INFO:homer.devices:Matched 1 device(s) for query 'asw2-c-eqiad.mgmt.eqiad.wmnet'
INFO:homer:Generating configuration for asw2-c-eqiad.mgmt.eqiad.wmnet
Configuration diff for asw2-c-eqiad.mgmt.eqiad.wmnet:
[edit interfaces interface-range disabled]
member ge-5/0/11 { ... }
+ member ge-5/0/12;
member ge-5/0/16 { ... }
[edit interfaces interface-range vlan-private1-c-eqiad]
- member ge-5/0/12;
[edit interfaces]
- ge-5/0/12 {
- description elastic1043;
- }
Type "yes" to commit, "no" to abort.
> yes
INFO:homer.transports.junos:Committing the configuration on asw2-c-eqiad.mgmt.eqiad.wmnet
INFO:homer:Homer run completed successfully on 1 devices: ['asw2-c-eqiad.mgmt.eqiad.wmnet']
Running Homer on asw2-b-eqiad.mgmt.eqiad.wmnet, it takes time ⏳ don't worry
INFO:homer.devices:Initialized 53 devices
INFO:homer:Committing config for query asw2-b-eqiad.mgmt.eqiad.wmnet with message: Host decommission - ryankemper@cumin1001 - T294805
INFO:homer:Gathering global Netbox data
INFO:homer.devices:Matched 1 device(s) for query 'asw2-b-eqiad.mgmt.eqiad.wmnet'
INFO:homer:Generating configuration for asw2-b-eqiad.mgmt.eqiad.wmnet
ERROR:homer:Device asw2-b-eqiad.mgmt.eqiad.wmnet failed to render the template, skipping.
Traceback (most recent call last):
File "/srv/deployment/homer/venv/lib/python3.7/site-packages/homer/netbox.py", line 46, in __getitem__
self.data[key] = getattr(self, method_name)()
File "/srv/deployment/homer/venv/lib/python3.7/site-packages/homer_plugins/wmf-netbox.py", line 164, in _get_junos_router_interfaces
jri[interface_name] = {**jri[parent], **interface_config}
UnboundLocalError: local variable 'parent' referenced before assignment
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/srv/deployment/homer/venv/lib/python3.7/site-packages/homer/__init__.py", line 299, in _execute
device_config.append(self._renderer.render(device.metadata['role'], device_data))
File "/srv/deployment/homer/venv/lib/python3.7/site-packages/homer/templates.py", line 57, in render
return template.render(data)
File "/srv/deployment/homer/venv/lib/python3.7/site-packages/jinja2/environment.py", line 1291, in render
self.environment.handle_exception()
File "/srv/deployment/homer/venv/lib/python3.7/site-packages/jinja2/environment.py", line 925, in handle_exception
raise rewrite_traceback_stack(source=source)
File "/srv/homer/public/templates/asw.conf", line 11, in top-level template code
{{ section("common", "vlans") }}
File "/srv/deployment/homer/venv/lib/python3.7/site-packages/jinja2/runtime.py", line 828, in _invoke
rv = self._func(*arguments)
File "/srv/homer/public/templates/macros/common.conf", line 3, in template
{% include category_name + "/" + section_name + ".conf" %}
File "/srv/homer/public/templates/common/vlans.conf", line 9, in top-level template code
{% if 'irb' in netbox.device_plugin.junos_router_interfaces.keys() -%}
File "/srv/deployment/homer/venv/lib/python3.7/site-packages/jinja2/environment.py", line 478, in getattr
return obj[attribute]
File "/srv/deployment/homer/venv/lib/python3.7/site-packages/homer/netbox.py", line 48, in __getitem__
raise HomerError(f'Failed to get key {key}') from e
homer.exceptions.HomerError: Failed to get key junos_router_interfaces
ERROR:homer:Homer run had issues on 1 devices: ['asw2-b-eqiad.mgmt.eqiad.wmnet']
Failed to run Homer on asw2-b-eqiad.mgmt.eqiad.wmnet
Traceback (most recent call last):
File "/srv/deployment/spicerack/cookbooks/sre/hosts/decommission.py", line 438, in run
switch, 'commit', str(self.reason)], check=True)
File "/usr/lib/python3.7/subprocess.py", line 487, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/usr/local/bin/homer', 'asw2-b-eqiad.mgmt.eqiad.wmnet', 'commit', 'Host decommission - ryankemper@cumin1001 - T294805']' returned non-zero exit status 1.
**Failed to run Homer on asw2-b-eqiad.mgmt.eqiad.wmnet**: Command '['/usr/local/bin/homer', 'asw2-b-eqiad.mgmt.eqiad.wmnet', 'commit', 'Host decommission - ryankemper@cumin1001 - T294805']' returned non-zero exit status 1.
ERROR: some step failed, check the task updates.
Updated Phabricator task T294805
END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) for hosts elastic[1039,1043].eqiad.wmnet
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL