Page MenuHomePhabricator

Drop the VarnishTrafficDrop and HAProxyEdgeTrafficDrop alerts
Closed, ResolvedPublic

Description

Deprecate the VarnishTrafficDrop and HAProxyEdgeTrafficDrop alerts, which were resulting in alert fatigue.

Further context: after discussion internally in the Traffic team, we have decided to retire these two alerts. The reason for doing so is because of the high number of false positives and the resulting alert fatigue. As an example, these alerts are sent out even when the site is depooled and in cases of traffic drops resulting from organic changes in traffic patterns. Pending further work on fixing and improving these alerts to negate such false positives, we decided that it's best to retire them and let other metrics such as NEL alert of such changes.

Event Timeline

Change 852206 had a related patch set uploaded (by Ssingh; author: Ssingh):

[operations/alerts@master] team-traffic: drop VarnishTrafficDrop and HAProxyEdgeTrafficDrop

https://gerrit.wikimedia.org/r/852206

ssingh triaged this task as Medium priority.Nov 2 2022, 2:43 PM
ssingh edited projects, added Traffic; removed Traffic-Icebox.

Change 852206 merged by Ssingh:

[operations/alerts@master] team-traffic: drop VarnishTrafficDrop and HAProxyEdgeTrafficDrop

https://gerrit.wikimedia.org/r/852206

BCornwall claimed this task.