Page MenuHomePhabricator

Make it possible to silence "high rate of matches" AbuseFilter notifications
Open, Needs TriagePublic

Description

In this task, let's create an option on saving an AbuseFilter that will allow the AbuseFilter maintainer to disable notifications for "high rate of matches". This is useful if we know that the high rate of matches is expected (e.g. T20110: Define AbuseFilter consequence to display a CAPTCHA and T372642: Create a "Challenge" type subtype of "Consequence").

Are there circumstances where we want to set $wmgEmergencyCaptcha across all projects without involving SREs? If not, then the AbuseFilter + showcaptcha consequence seems like it would suffice for the use case described in this task.

This was discussed in today's WMF-Steward call.

An outstanding issue with AbuseFilter that $wmgEmergencyCaptcha was recently enabled for was to prevent filter maintainers from being spammed with notifications that the filter was matching a large volume of edits. Rate limiting those notifications (maybe max 1/filter/hour?) or being able to disable them would allow us to decline this task.

Event Timeline

Rate limiting those notifications (maybe max 1/filter/hour?) or being able to disable them would allow us to decline this task.

Yes please - so far I've received 23 notifications/emails for a filter I just emergency-enabled which deliberately matches a large % of edits.

I had a look at how these notifications are produced, in case it's useful before picking between the two options.

EmergencyWatcher already tries to notify only once: getFiltersToThrottle skips any filter where isThrottled() is true. So 23 notifications for one filter suggests something is defeating that guard rather than the guard being absent.

Two candidates. First, af_throttled = 1 is written in a DeferredUpdates::addUpdate, and notifyForFilter runs in a separate deferred callback. Requests that land before that write commits all see isThrottled() as false, so a burst of matches can queue a burst of notifications from what is really one throttle event. Second, FilterStore line 101 keeps the throttled flag only when the filter is being disabled, so re-enabling a filter clears it. Enabling a broad filter, watching it throttle, and enabling it again will notify each time, which fits the case described here.

If the burst is mostly the first one, then deduplicating would remove most of the pain without any new UI, and the per-filter opt-out could stay scoped to the CAPTCHA case in T303433 where high match rates are genuinely expected. If it is mostly the second, an opt-out is the better fit.

I have not written a test to confirm the race, so treat the first one as a hypothesis. Happy to dig further or take this on if that would help.

Change #1325929 had a related patch set uploaded (by Rajveer42; author: Rajveer42):

[mediawiki/extensions/AbuseFilter@master] EchoNotifier: only notify once per throttled filter

https://gerrit.wikimedia.org/r/1325929