We had a discussion today at the weekly SRE infra foundations meeting about promoting fastnetmon "Possible DDOS" notifications from email to something more visible (IRC? Paging?). This led to a discussion about escalating network alerts to improve visibility in general. However, consensus was not yet reached about how to best implement noisier alert notifications from something that isn't icinga.
Creating this tracking task to further explore options to improve visibility of important network alerts and implement something in the near-term, with the understanding that larger scale efforts are under way to optimize alert escalation/notification for the long-term, and that these alerts will eventually be migrated to that flow. In other words, focusing on the low hanging fruit.