Page MenuHomePhabricator

AbuseFilter throttled filter error message is confusing – please update the copy
Closed, ResolvedPublic5 Story Points

Description

On
https://pt.wikipedia.org/wiki/Special:AbuseFilter/70?uselang=en
There is the following line

Flags: Warning: This filter was automatically disabled as a safety measure. It reached the limit of matching more than 5.00% of actions.

But a few lines above it has this:

Statistics: Of the last 3,518 actions, this filter has matched 2 (0.06%). On average, its run time is 1.99 ms, and it consumes 288 conditions of the condition limit.

Obviously, 0.06% < 5.00%, so one of the message is wrong.

Besides, the latest edit to that filter was on 20:33, 15 July 2013, but there is no timestamp indicating when that warning was added. The filter's log has many (correct) detections since the last edit was made:
https://pt.wikipedia.org/wiki/Special:AbuseLog?dir=prev&wpSearchFilter=70&offset=20130715203331&uselang=en
and the option "Enable this filter" is still marked ([x] instead of [ ]), indicating it is indeed enabled.

If the extension indeed disabled the filter, but enabled it again automatically at some point, it should say so. And if it didn't disable the filter, it should not say it is disabled.

The current situation is confusing.


Version: unspecified
Severity: major


Acceptance criteria

Details

Reference
bz52525

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 1:51 AM
bzimport added a project: AbuseFilter.
bzimport set Reference to bz52525.
bzimport added a subscriber: Unknown Object (MLST).
He7d3r created this task.Aug 4 2013, 4:20 PM

I edited that filter yesterday[1], and now the message is gone. Whatever the underlying problem is, which makes the filter not to be disabled, at least the interface is updated once we edit the filter (but the message shouldn't be displayed in the first place, if the filter is kept enabled...)

[1] https://pt.wikipedia.org/wiki/Special:AbuseFilter/history/70?uselang=en

This happened again with another filter, per
https://pt.wikipedia.org/w/index.php?diff=37362443

He7d3r updated the task description. (Show Details)Jan 22 2015, 9:42 PM
He7d3r added a project: Regression.
He7d3r set Security to None.
Restricted Application added a subscriber: Luke081515. · View Herald TranscriptJul 6 2015, 3:17 PM

Yeah, AbuseFilter is lying – it doesn't disable the whole filter, it only prevents it from doing "restricted" actions – $wgAbuseFilterRestrictedActions = array( 'block', 'degroup', 'blockautopromote', 'rangeblock' );. I think that part works, it's the message text that's very misleading.

This will be set if the limit of actions is met in the temporary statistics that AbuseFilter keeps, and will only be cleared when you disable and re-enable the filter (or make any other change to it).

MarcoAurelio added a comment.EditedJul 26 2017, 1:40 PM

This is still happening on 2017 (this ticket is from 2013) without any action taken to resolve it. Is someone interested in working on a patch to fix this?

As for T173250: Automatically disabled AbuseFilter rule is not really disabled on azb.wikipedia.org, there the statistics and the message matched. It was matching 12% of edits and claimed to have been disabled due to matching more than 5% (which seems right). Except that, in reality, it was not disabled and was still preventing edits from being saved (not warn-only).

TBolliger set the point value for this task to 5.Aug 17 2017, 7:39 PM
dbarratt claimed this task.Aug 22 2017, 2:35 PM
dbarratt moved this task from Ready to In progress on the Anti-Harassment (AHT Sprint 3) board.
dbarratt removed dbarratt as the assignee of this task.Aug 22 2017, 6:03 PM
dbarratt moved this task from In progress to Ready on the Anti-Harassment (AHT Sprint 3) board.
dbarratt added a subscriber: dbarratt.
Huji added a subscriber: Huji.Aug 23 2017, 4:44 PM

Not sure if related or not, but in a fresh installation of AbuseFilter without object caching (i.e. no throttling), the filter does get throttled after being hit a few times.

dmaza claimed this task.Aug 24 2017, 5:18 PM
dmaza moved this task from Ready to In progress on the Anti-Harassment (AHT Sprint 3) board.

Here are my findings.

Filters never get disabled, they get throttled. In this context, throttled means that the filter will ignore certain actions if it is configured to do so with $wgAbuseFilterRestrictions (warn, disallow, etc), otherwise, even if it is throttled the filter prevent edits or whatever it is meant to do.

The only way to remove the throttled flag is by editing the filter. Any edit will do it if you keep the filter enabled.

The disconnection between

Flags: Warning: This filter was automatically disabled as a safety measure. It reached the limit of matching more than 5.00% of actions.

and

Statistics: Of the last 3,518 actions, this filter has matched 2 (0.06%). On average, its run time is 1.99 ms, and it consumes 288 conditions of the condition limit.

happens 'cause filters don't gets unthrottled dynamically and statistics keep changing.

@TBolliger I'm not sure what to do here. Should I update the message to reflect what I said above or do we want to discuss a different behavior for how throttled/disabled should work?

This comment was removed by TBolliger.

Great investigation, Dayllan. Thank you.

I propose we change the scope of this ticket to updating the warning message to something along the lines of Flags: Warning: The action(s) of this filter were automatically disabled as a safety measure. It reached the limit of matching more than 5.00% of actions. Feedback on this wording is encouraged and appreciated.

@dmaza has some ideas on how to build a better throttling/disabling functionality for AbuseFilter. Can you please create a Phab ticket to track your ideas, and share the link here? Thank you!

dmaza added a comment.Aug 29 2017, 5:11 PM

What about this? Removing the threshold on the message might avoid the confusion.

Flags: Warning: This filter was automatically throttled as a safety measure and it's actions might be disabled.

Lahwaacz added a comment.EditedAug 29 2017, 5:22 PM

I think that is still confusing given the current state of the AbuseFilter documentation.

@dmaza said:

In this context, throttled means that the filter will ignore certain actions if it is configured to do so with $wgAbuseFilterRestrictions (warn, disallow, etc), otherwise, even if it is throttled the filter prevent edits or whatever it is meant to do.

So the warning message should say which actions might be disabled and the documentation should say how $wgAbuseFilterRestrictions is involved in throttling. This table currently says only:

$wgAbuseFilterRestrictions
array( 'block' => true, 'degroup' => true, 'blockautopromote' => true, 'rangeblock' => true )
Users must have the "abusefilter-modify-restricted" user right as well as "abusefilter-modify" in order to create or modify filters which carry out these actions.

The message (whatever it ends up being) should only be shown if the filter actually has any restricted actions it can perform.

The message (whatever it ends up being) should only be shown if the filter actually has any restricted actions it can perform.

Yes.

TBolliger renamed this task from AbuseFilter doesn't disable filters for safety measure, but says it does! to AbuseFilter throttled filter error message is confusing – please update the copy.Aug 29 2017, 6:02 PM
TBolliger updated the task description. (Show Details)
TBolliger updated the task description. (Show Details)Aug 29 2017, 9:18 PM

@dmaza — Were you going to share a screenshot of the error?

I like Flags: Warning: This filter was automatically throttled as a safety measure and it's actions might be disabled.

Nirmos added a subscriber: Nirmos.Aug 29 2017, 9:48 PM

Flags: Warning: This filter was automatically throttled as a safety measure and it's actions might be disabled

This is still very vague ("might be disabled").

it's (it is) → its (possessive)

"Flags: Warning:" is this hierarchy really needed? Are the flags (that are visible to the end user) that are not warnings?

It also does not answer the question "what do I have to do to reset it?".

Actually, I think we can scratch one of those points. The "Flags:" prefix seems to have been removed since this task was filed.

Potentially related: T86088: AbuseFilter should provide an easy way to un-throttle a rule. — but we may not want this at all, given it would encourage users to re-enable an unoptimized filter without fixing what's wrong.

This filter was automatically throttled as a safety measure and the following actions will not execute (disallow|tag). Please review and [[mw:Extension:AbuseFilter/Conditions|optimize]] your conditions.

@MusikAnimal — Please chime in. :)

What about the following message?

This filter was automatically throttled as a safety measure and the following actions will not execute (disallow|tag). Please review and [[mw:Extension:AbuseFilter/Conditions|optimize]] your conditions before editing the filter to unthrottle it

The message (whatever it ends up being) should only be shown if the filter actually has any restricted actions it can perform.

If none of the filter actions are part of wgAbuseFilterRestrictions the message will not show, 'cause technically it is not being throttled

Well, I've learned a lot from reading this task! I was under the impression the filter was disabled in its entirety, no thanks to the misleading message.

Filters never get disabled, they get throttled. In this context, throttled means that the filter will ignore certain actions if it is configured to do so with $wgAbuseFilterRestrictions (warn, disallow, etc), otherwise, even if it is throttled the filter prevent edits or whatever it is meant to do.

I'm still confused. "Throttled" means it won't ever do the restricted actions (until the throttle flag is removed by altering the filter in some way)? Why is the term "throttled" being used? Instead, it sounds like it's disabling the restricted actions, no? If it is throttling in a literal sense, to what degree?

As for T173250: Automatically disabled AbuseFilter rule is not really disabled on azb.wikipedia.org, there the statistics and the message matched. It was matching 12% of edits and claimed to have been disabled due to matching more than 5% (which seems right). Except that, in reality, it was not disabled and was still preventing edits from being saved (not warn-only).

This is a great example of what we don't want to happen. Do we know what $wgAbuseFilterRestrictions is set to for this wiki? If "disallow" is not one of them, I think it should be, and be the default for all wikis.

I'm still confused. "Throttled" means it won't ever do the restricted actions (until the throttle flag is removed by altering the filter in some way)? Why is the term "throttled" being used? Instead, it sounds like it's disabling the restricted actions, no? If it is throttling in a literal sense, to what degree?

I don't think "throttle" is the correct verb, actually. "Placed in "Safety mode"" may be better, since I'm not sure there is an existing verb that describes this type of behavior.

This is a great example of what we don't want to happen. Do we know what $wgAbuseFilterRestrictions is set to for this wiki? If "disallow" is not one of them, I think it should be, and be the default for all wikis.

I created T174624: Audit and set smarter defaults for wgAbuseFilterRestrictions to track this. Please edit and expand as you see fit.

dmaza added a comment.Aug 31 2017, 2:21 PM

I'm still confused. "Throttled" means it won't ever do the restricted actions (until the throttle flag is removed by altering the filter in some way)? Why is the term "throttled" being used? Instead, it sounds like it's disabling the restricted actions, no? If it is throttling in a literal sense, to what degree?

Throttled is not the right verb. Nothing gets throttled in the literal sense. I'm using "Throttled" 'cause that's the name of the flag. What this flag does is that restricted actions don't get executed.

This is a great example of what we don't want to happen. Do we know what $wgAbuseFilterRestrictions is set to for this wiki? If "disallow" is not one of them, I think it should be, and be the default for all wikis.

I don't know the defaults for $wgAbuseFilterRestrictions. We can deal with that in T174624: Audit and set smarter defaults for wgAbuseFilterRestrictions.

dmaza added a comment.Aug 31 2017, 2:23 PM

Regarding this task, will the following message make it more clear for users when filters get "Throttled"? Like @MusikAnimal pointed out, using the verb throttled doesn't make sense:

This filter was automatically flagged as harmful. As a safety measure, the following actions will not execute (disallow|tag). Please review and [[mw:Extension:AbuseFilter/Conditions|optimize]] your conditions before editing the filter to remove this restriction

Please review and [[mw:Extension:AbuseFilter/Conditions|optimize]] your conditions before editing the filter to remove this restriction

Is it possible to optimize the conditions without editing the filter to remove the restriction? If not, "before" is not the right word. (And removing the flag on any edit is probably not the best mechanism.)

dmaza added a comment.Aug 31 2017, 3:58 PM

@Lahwaacz It is not possible to optimize without saving the filter. I also agree that removing the flag on every edit is not optimal. I think filters that are considered harmful should be disabled but that's out of the scope here.

What about this then?

This filter was automatically flagged as harmful. As a safety measure, the following actions will not execute (disallow|tag). Please review and [[mw:Extension:AbuseFilter/Conditions|optimize]] your conditions to remove this restriction

This implies that you have to edit the conditions (for better or worst) if you want to unthrottle the filter.

dmaza added a comment.Sep 5 2017, 2:14 PM

@TBolliger @MusikAnimal We need to make a decision on this. Any thoughts on my last message?

We need to make a decision on this. Any thoughts on my last message?

Your proposed copy sounds good to me :)

We need to make a decision on this. Any thoughts on my last message?

Your proposed copy sounds good to me :)

Agreed. 👍

Change 376091 had a related patch set uploaded (by Dmaza; owner: Dmaza):
[mediawiki/extensions/AbuseFilter@master] Fix confusing warning message on throtthled filters

https://gerrit.wikimedia.org/r/376091

Change 376091 merged by jenkins-bot:
[mediawiki/extensions/AbuseFilter@master] Fix confusing warning message on throtthled filters

https://gerrit.wikimedia.org/r/376091

TBolliger closed this task as Resolved.Sep 18 2017, 6:28 PM