Page MenuHomePhabricator

Filters are malfunctioning, supplying wrong values for variables, not working as they should
Open, HighPublic

Description

Over and over we're noticing more filters are malfunctioning. It seems that the variables AbuseFilter supplies are incorrect, or it's reading the data in the wrong way, something related to the variables...

T173569 - user_groups randomly doesn't work
T168736 - variables have the wrong values
T173977 - the complete wrong edits are being tripped, the abuse log and the diffs contradict each other

And today we got this report where a perfectly crafted filter did not trip edits that it should have. On English Wikipedia we have 0% of filters exceeding the condition limit, so there's no excuse for this filter to not work.

Fortunately it doesn't look like these issues are widespread, but they are letting some bad edits get through while logging good edits when they shouldn't be. I'm not aware of any disallowing filters that are malfunctioning, but it seems we can't rule that out given the unpredictability of the extension.

I suspect something changed earlier this summer that introduced all these seemingly related bugs. All three of the above tasks have received no attention from maintainers. I'm very unfamiliar with the codebase but I'm going to try to dig through the git log and see if anything stands out as a possible culprit, but any help is greatly appreciated. Many thanks!

Event Timeline

Restricted Application added subscribers: Liuxinyu970226, Jay8g, TerraCodes, Aklapper. · View Herald TranscriptSep 14 2017, 4:59 PM

Pinging @matej_suchanek who I know has made a lot of changes recently. Definitely not pointing the finger but perhaps you have some insight as to what is going on?

I have focused on interface bugs, so I would be surprised if that was me (but everything is possible ;)

I thoroughly read the linked tasks but I'm afraid I cannot help with any of those. Filters seem to mostly work and the problems are rare, occurring randomly. Since I'm not a maintainer anywhere but on cswiki and Wikidata, I cannot look into those logs. Niether do I have developer access that would allow me to see private logs.

Anyway, my personal observation is that retrospective testing is not that reliable to help with fixing an internal bug.

greg added subscribers: hoo, greg.Sep 18 2017, 10:52 PM

Adding @hoo to this UBN! task as the maintainer of AbuseFilter per https://www.mediawiki.org/wiki/Developers/Maintainers

greg added a comment.Sep 25 2017, 11:29 PM

Adding @hoo to this UBN! task as the maintainer of AbuseFilter per https://www.mediawiki.org/wiki/Developers/Maintainers

Ping.

Status?

Got another report to today....

Filter 139 (fixed position vandalism) failed to disallow the edits by this account when it should have. You can go to Special:AbuseFilter/test/139 and plug in their username and see that the edits match (you may need to be a sysop since the edits were revision-deleted).

This instance is a bit more serious, as the vandalism was on a template that was transcluded on over 1,000 articles.

Today Special:AbuseFilter/852 was not tripped with this edit. Again you can use the debug tools and see that it matches.

Just noting this in case it helps figure out what's going wrong. The only thing filters 139 and 852 have in common is that they read added_lines. Filter 812 (mentioned in the task description) however does not use added_lines, so my guess is the issue is not tied to the usage of any specific variables or functions.

He7d3r added a subscriber: He7d3r.Oct 28 2017, 3:04 PM
greg added a comment.Nov 2 2017, 8:50 PM

Meta: Is this task meant to track some overarching work? It mentions three tasks that look like they could/should be sub-tasks. If so, this should not be UBN! for multiple months. Is "High" sufficient for the needs?

MusikAnimal lowered the priority of this task from Unbreak Now! to High.Nov 2 2017, 10:22 PM

Sorry, I don't mean to misuse UBN. This was a cry for help because for so many moons no attention was given to all of these high-priority bugs. Similar bugs continue to occur, I just haven't been creating tasks for them because I think they're probably all related – hence the parent UBN task. I know the Anti-harassment Tools Team is working on some of this, and the fix(es) certainly won't happen overnight, so if we want to change to High priority that is fine :)

To clarify, there are six different reports here of filters malfunctioning:

It wasn't until earlier this year (June 2017, when T168736 was filed) that I have noticed such unpredictable and ongoing malfunctioning, so I still stand by my belief that there was some sort of regression or external change that is causing all of this. Just a theory, though!

greg removed a subscriber: greg.Nov 3 2017, 5:03 PM

Cool, thanks for clarifying! I'll assume the Anti-harassment Tools Team will prioritize this work accordingly and now remove this from my watchlist (well, it automatically as it's not UBN! anymore ;) ).

New report:

Filter 884 @ https://en.wikipedia.org/wiki/Special:AbuseFilter/history/884/item/18025 should have disallowed this edit. Same as before, you can batch test and see that it matches, it just mysteriously didn't disallow or even flag the edit when it was made. However it did correctly disallow this edit, which was made by the same user.

MusikAnimal added a comment.EditedNov 15 2017, 3:50 PM

These edits (1 2, now revision-deleted) should have been disallowed by 139.

Assuming you are able to view revision-deleted edits (admin, steward, staff, etc.), go to Special:AbuseFilter/test and put in Mikaylajackson as "Changes by user". The edits in question are shown.

Struck... the filter was working correctly.

We hit a wall in our last investigation of these defects. We'll look again in this sprint or our next.

Nihlus added a subscriber: Nihlus.Dec 1 2017, 9:36 PM
Samtar added a subscriber: Samtar.Dec 1 2017, 9:40 PM

Another report, not safe for work (sorry) and revision deleted [ diff ], should have been caught by 139.

Safer to use https://en.wikipedia.org/wiki/Special:AbuseFilter/test/139 against user Dimetappir

Another report, not safe for work (sorry) and revision deleted [ diff ], should have been caught by 139.
Safer to use https://en.wikipedia.org/wiki/Special:AbuseFilter/test/139 against user Dimetappir

I know why this and the edits reported at T175933#3763292 got through. Per WP:BEANS I'm not going to say anything, but for anyone working on this, just know that these are not false positives, and that the filter was working correctly.

So, do we have any update for this? I closed the user_groups issue since it seems not to happen anymore, but here it's difficult to tell what is a bug and what isn't. All these troubles seems to be strictly related, they may also have the same cause. However, we first need to precisely tell what is (still) wrong, gather fresh examples, figure out how to reproduce that and only then we may think of a fix.

Daimona moved this task from Backlog to Internal bugs on the AbuseFilter board.Apr 3 2018, 12:19 PM