Yesterday I reset all the rabbit nodes and rebuilt all queues. I did this knowing it would cause openstack services to crash, but I was on call and kept an eye on alert manager, silencing each random alert as it came up.
Despite my rapid silencing of things, some pages still got escalated past me and forwarded to the rest of the wmcs team. For example, 'cloudvirt-wdqs1001/nova-compute prox maximum'
Clearly I should have silenced things more thoroughly before I started, and this issue is likely just me not understanding how on-call/paging works. But... shouldn't acking things in alert manager prevent paging?