Page MenuHomePhabricator

Silencing alerts through the alerts dashboard is supported and functional
Closed, ResolvedPublic

Description

This task tracks enabling silences on https://alerts.wikimedia.org and implementing "acknowledgments" via AM silences (with https://github.com/prymitive/kthxbye)

  • Enable silences on alerts.w.o
  • Deploy kthxbye as a singleton and enable alerts.w.o support for acks

Event Timeline

Change 638044 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/debs/kthxbye@master] Add Debian directory

https://gerrit.wikimedia.org/r/638044

Change 638998 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] alertmanager: add ack daemon

https://gerrit.wikimedia.org/r/638998

Change 638999 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] alertmanager: enable acks and silences on alerts.w.o

https://gerrit.wikimedia.org/r/638999

Change 639000 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] profile: add prometheus jobs for am acks

https://gerrit.wikimedia.org/r/639000

Change 638044 merged by Filippo Giunchedi:
[operations/debs/kthxbye@master] Add Debian directory

https://gerrit.wikimedia.org/r/638044

Change 638998 merged by Filippo Giunchedi:
[operations/puppet@production] alertmanager: add ack daemon

https://gerrit.wikimedia.org/r/638998

Change 638999 merged by Filippo Giunchedi:
[operations/puppet@production] alertmanager: enable acks and silences on alerts.w.o

https://gerrit.wikimedia.org/r/638999

Change 639000 merged by Filippo Giunchedi:
[operations/puppet@production] profile: add prometheus jobs for am acks

https://gerrit.wikimedia.org/r/639000

Status update: alerts now can be acknowledged via silences by prefixing the silence text with ACK!

fgiunchedi claimed this task.

Change 659233 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] alertmanager: have kthxbye listen on all addresses by default

https://gerrit.wikimedia.org/r/659233

Change 659233 merged by Filippo Giunchedi:
[operations/puppet@production] alertmanager: have kthxbye listen on all addresses by default

https://gerrit.wikimedia.org/r/659233