Page MenuHomePhabricator

(Re) evaluate effectiveness / usefulness of varnish/haproxy traffic drop alerts
Closed, ResolvedPublic

Description

I (Filippo) have seen these alerts fire reasonably frequently (both in -operations and -traffic), but hardly ever be actionable in an meaningful way (except maybe look at dashboards). They also fire during high traffic incidents, obviously after the peak has passed and there's now a "drop" in traffic. For these reasons I think we should re-evaluate the alerts and see if they are still effective, useful, wanted, etc.

Event Timeline

I know that I ignore them. Perhaps rather than removing them entirely, we could tweak the detection to be a little smarter with detecting serious drops (i.e. falling off a cliff rather than just dipping suddenly).

akosiaris subscribed.

Removing SRE, has already been triaged to a more specific SRE subteam

fgiunchedi claimed this task.

I'm boldly resolving the task since AFAIK the traffic drop alerts have been removed, see also discussion at https://gerrit.wikimedia.org/r/c/operations/alerts/+/900626