As discussed on IRC:
The check_ferm_active.prom file is stale on a host and restarting the fern service and the nrpe2nodexp-ferm_active.service service did not help.
The timer related to nrpe2nodexp-ferm_active.service is active.
ssh es2049.codfw.wmnet sudo ls -ltr /var/lib/prometheus/node.d/check_ferm_active.prom -rw-r--r-- 1 nagios prometheus-node-exporter 952 Aug 29 13:25 /var/lib/prometheus/node.d/check_ferm_active.prom
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| mysqld_exporter.pp: reset /var/log/prometheus perms | operations/puppet | production | +2 -1 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | FCeratto-WMF | T402859 Productionize es2049-es2057 | |||
| Open | tappof | T403615 Stale check_ferm_active.prom file |
Change #1184544 had a related patch set uploaded (by Federico Ceratto; author: Federico Ceratto):
[operations/puppet@production] mysqld_exporter.pp: fix /var/log/prometheus perms