Condition limit is broken: consumes 100,000+ conditions limit is set to 1000
OpenPublic

Description

For a extreme example, consider this filter:
https://en.wikipedia.org/wiki/Special:AbuseFilter/473

How can it consumes 23296 conditions if the condition limit is set to 1000?

Why does that value changes every second?
(reloading the page a few times I got: 51172, then 3986, then 7859, 14690, and so on...)

This is very confusing for users trying to debug and optimize the existing filters...


Version: unspecified
Severity: normal
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=45045
https://bugzilla.wikimedia.org/show_bug.cgi?id=41691
https://bugzilla.wikimedia.org/show_bug.cgi?id=55459

bzimport added a subscriber: wikibugs-l.
bzimport set Reference to bz51294.
He7d3r created this task.Via LegacyJul 13 2013, 4:14 PM
He7d3r added a comment.Via ConduitJul 13 2013, 4:19 PM

I don't know if this is a problem, but I noticed the function getFilterProfile uses three keys:
wfMemcKey( 'abusefilter', 'profile', $filter, 'count' );
wfMemcKey( 'abusefilter', 'profile', $filter, 'total' );
wfMemcKey( 'abusefilter', 'profile-conds', 'total' );

But the resetFilterProfile only resets the first two. Shouldn't it also reset the key profile-conds?

[1] https://git.wikimedia.org/blob/mediawiki%2Fextensions%2FAbuseFilter.git/HEAD/AbuseFilter.class.php#L535

Aklapper added a comment.Via ConduitJul 15 2013, 11:16 AM

(In reply to comment #0)

For a extreme example, consider this filter:
https://en.wikipedia.org/wiki/Special:AbuseFilter/473

How can it consumes 23296 conditions if the condition limit is set to 1000?

Where exactly can I see that (position in the UI)?

He7d3r added a comment.Via ConduitJul 15 2013, 11:48 AM

There is a line like this:

Statistics: Of the last 147 actions, this filter has matched 4 (2.72%). On average, its run time is 1.77 ms, and it consumes 13,105 conditions of the condition limit.

(the text comes from [[MediaWiki:abusefilter-edit-status]])

He7d3r added a comment.Via ConduitJul 15 2013, 1:02 PM

It really doesn't known what a "limit" is: I just saw it saying filter 473 consumes 112,686 conditions (!)

werdna removed a subscriber: werdna.Via WebDec 10 2014, 5:18 PM
Dragons_flight added a subscriber: Dragons_flight.Via WebFeb 25 2015, 12:39 AM

This is really three separate bugs that all lead to per filter stats corruption:

  1. Failure to reset properly when a filter is edited.
  2. Failure to count properly when evaluation is aborted due to the condition limit being reached.
  3. A race condition that causes near simultaneous instances of the AbuseFilter to corrupt each others stats.

The third problem is the dominant one for the highly active wikis and can lead to really silly results. Both per filter condition counts and run times are affected, though the nonsensical condition counts are more noticeable.

I am nearly done with a patch that addresses this bug.

gerritbot added a subscriber: gerritbot.Via ConduitWed, Apr 1, 4:54 AM

Change 201104 had a related patch set uploaded (by Dragons flight):
Overhaul AbuseFilter internal profiling system

https://gerrit.wikimedia.org/r/201104

gerritbot added a project: Patch-For-Review.Via ConduitWed, Apr 1, 4:54 AM

Add Comment

Column Prototype
This is a very early prototype of a persistent column. It is not expected to work yet, and leaving it open will activate other new features which will break things. Press "\" (backslash) on your keyboard to close it now.