Of the last 3,003 actions, 2 (0.07%) have reached the condition limit of 1,000, and 25 (0.83%) have matched one of the filters currently enabled.
Can we have the $wgAbuseFilterConditionLimit changed. +500 conditions?
We have to run a lot of powerful filters on commons to prevent abuse.
I also checked all enabled filters on commons, and as far i can see we need them all.
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| Set $wgAbuseFilterConditionLimit = 2000 for commonswiki | operations/mediawiki-config | master | +1 -0 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | matmarex | T132048 Change $wgAbuseFilterConditionLimit for wikimedia commons | |||
| Resolved | matmarex | T132200 Enable $wgAbuseFilterProfile on Commons for a few days | |||
| Resolved | matmarex | T132189 Restore display of number of conditions used by filter on the filter's page, gated by new config variable $wgAbuseFilterProfile |
(Claiming this task as a reminder to myself to review the filters after T132200, I'm hoping we'll end up marking this "invalid". :) )
I took a look today and I'm confident that this won't be necessary. :D
Commons is now safely below the condition limit without any filter changes, due to my recent changes to the way conditions are counted (T43693 and T132190). These made the number of conditions used by a filter lesser in almost all cases (and closer to the truth, too; see some examples in the commit message of https://gerrit.wikimedia.org/r/#/c/282477/).
There are 63 enabled filters on Commons, and according to the data, almost every one of them executes in under 1 ms (on average) and consumes only 1 or 2 conditions (on average). Most are closer to 0.15 ms and use just one condition. There are a few outliers, I'll look into them some more.
I looked at some slower filters, and I have two observations that seem not to be documented anywhere prominent (at least not at https://www.mediawiki.org/wiki/Extension:AbuseFilter/Conditions).
I edited the following filters (see https://commons.wikimedia.org/wiki/Special:AbuseFilter/history) – note that some are private and only Commons sysops will be able to see the diffs:
(I also made another mistaken edit to filter 69, which I reverted.)
I tried not to touch filters where the improvements weren't obvious. There seemed to be many where user_groups or user_editcount is used in the first condition, but they all looked very quick anyway.
@matmarex: I didn't get the reasoning behind the edit to filter 69:
(2016-04-17) Reordered rules for performance. The checks for user_name/global_user_groups are almost never hit, so putting them first doesn't let us skip the regexes. Let's just put the regexes first. --Matma Rex
If they are almost never hit, isn't better for short circuit purposes to let them as first conditions? Isn't it a problem that the (new|old)_wikitext can be quite large and will now be tested more often against the regexes?
What I mean is that e.g. "OTRS-member" in global_user_groups is almost never true (most users are not OTRS members); !("OTRS-member" in global_user_groups) (negated, as it appears in the filter) is almost always true and doesn't let us short-circuit the filter. Perhaps I should have said that they are almost always hit?
Again:
Of the last 3,574 actions, 1 (0.03%) has reached the condition limit of 1,000, and 22 (0.62%) have matched one of the filters currently enabled.
:-(
I'd suggest to disable some infrequently used filters:
The action check could be put first in some of these:
Some more work along the lines of what MatmaRex said above is in order, although it would be useful to know which filters are consuming most conditions.
And i disabled 107.
But it wouldn't hurt to have a bit moor conditions :-). Commons has a small community, and thus we need more filters than other wikis.
Change 294363 had a related patch set uploaded (by Bartosz Dziewoński):
Set $wgAbuseFilterConditionLimit = 2000 for commonswiki
Change 294363 merged by jenkins-bot:
Set $wgAbuseFilterConditionLimit = 2000 for commonswiki
Mentioned in SAL [2016-06-14T23:09:45Z] <ori@tin> Synchronized wmf-config/abusefilter.php: I4e5e4d227: Set $wgAbuseFilterConditionLimit = 2000 for commonswiki (T132048) (duration: 00m 28s)
For the record, the condition limit has been recently increased to 2000 for all wikis: T309609.