Page MenuHomePhabricator

Icinga BFD check failing
Closed, ResolvedPublic

Description

Noticed in Icinga the BFD check was failing for at least one device. Very strange, failing to load the MIB it seems:

	Traceback (most recent call last):
File "/usr/lib/nagios/plugins/check_bfd.py", line 70, in <module>
main()
File "/usr/lib/nagios/plugins/check_bfd.py", line 28, in main
load("BFD-STD-MIB")
File "/usr/lib/python3/dist-packages/snimpy/manager.py", line 595, in load
m = mib.load(mibname)
^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/snimpy/mib.py", line 656, in load
raise SMIException("unable to find {} (check the path)".format(mib))
snimpy.mib.SMIException: unable to find b'BFD-STD-MIB' (check the path)

Event Timeline

cmooney triaged this task as Medium priority.Mar 5 2024, 5:09 PM
cmooney created this task.

Affecting also the cloudswitches

image.png (506×520 px, 53 KB)

I've bandaided the issue on alert2001, we'll need a more proper fix:

# download-mibs
# cd /var/lib/snmp && ln -s ../mibs

There is this package on the alert hosts:

ii  snmp-mibs-downloader                 1.2                             all          install and manage Management Information Base (MIB) files

What is looking for the missing MIB file is the Snimpy "load" (https://snimpy.readthedocs.io/en/latest/usage.html)

I guess the snmp-mibs-downloader just has to be automated to download stuff?

It was missing this one: https://www.circitor.fr/Mibs/Html/B/BFD-STD-MIB.php

I guess the snmp-mibs-downloader just has to be automated to download stuff?

Yeah on it's own that package installs but doesn't do anything. Need to somehow trigger the command if the MIBs aren't present on a new build I think.

Looks like it's:

man 1 download-mibs
download-mibs --help

and the config is at /etc/snmp-mibs-downloader/snmp-mibs-downloader.conf which has some kind of "AUTOLOAD" config line, fwiw.

Looks like it's:

man 1 download-mibs
download-mibs --help

and the config is at /etc/snmp-mibs-downloader/snmp-mibs-downloader.conf which has some kind of "AUTOLOAD" config line, fwiw.

Sorry yes I may be mistaken. I know the default behaviour if installed does nothing, but we may indeed have specific config to tell it to do it automatically.

Looks like @fgiunchedi kicked it off manually with the downlad-mibs command, but perahps that wasn't needed and the location was the only real issue.

Change 1008941 had a related patch set uploaded (by Andrea Denisse; author: Andrea Denisse):

[operations/puppet@production] icinga: Set explicit BASEDIR for MIB database in snmp-mibs-downloader

https://gerrit.wikimedia.org/r/1008941

Change #1014483 had a related patch set uploaded (by Filippo Giunchedi; author: Filippo Giunchedi):

[operations/puppet@production] snmp: instruct libsmi with snmp-mibs-downloader path

https://gerrit.wikimedia.org/r/1014483

Change #1014483 merged by Filippo Giunchedi:

[operations/puppet@production] snmp: instruct libsmi with snmp-mibs-downloader path

https://gerrit.wikimedia.org/r/1014483

This is fixed, I've undone my symlink bandaid. I've also reported the issue at https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1067768

Change #1008941 abandoned by Andrea Denisse:

[operations/puppet@production] icinga: Set explicit BASEDIR for MIB database in snmp-mibs-downloader

Reason:

Abandoning in favor of 1014483

https://gerrit.wikimedia.org/r/1008941