Page MenuHomePhabricator

Prevent depooled Prometheus to send alerts during maintenance
Closed, ResolvedPublic


During the latest switch maintenance (T329073) prometheus1005 was depooled from LVS, though not "depooled" from Alertmanager, in the sense that the host kept firing alerts (from its POV anyways). We should be more proactive and make sure we can effectively prevent a depooled host from sending alerts too during maintenance

Related Objects

Event Timeline

Change 900238 had a related patch set uploaded (by Filippo Giunchedi; author: Filippo Giunchedi):

[operations/puppet@production] DNM: test alertmanager depool for prometheus1006

Setting alertmanagers: [] for the host in question is enough to remove its AM configuration, see also this PCC