Allow search through abuse filter descriptions and pattern
Closed, ResolvedPublic

Description

I have not found a special page which allows a plain or regex search through all pattern or description for the wiki.
This can be helpful when searching for a specific page name or part of its or for user names or ...

Should get options for enabled/disabled, private/non-private, deleted/not deleted and for meta: global/local

At the moment the only way is using the api to see all the pattern and use the browser search

https://de.wikipedia.org/w/api.php?action=query&list=abusefilters&abfprop=id%7Cpattern&abflimit=max

Umherirrender updated the task description. (Show Details)
Umherirrender raised the priority of this task from to Needs Triage.
Umherirrender added a project: AbuseFilter.
Umherirrender added a subscriber: Umherirrender.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 23 2015, 8:12 PM
Aklapper triaged this task as Lowest priority.Jan 24 2015, 2:07 AM
Huji added a subscriber: Huji.Feb 10 2018, 2:13 PM
Daimona added a comment.EditedFeb 15 2018, 6:45 PM

I'm trying to work on this, in such a way that the search field is in the AbuseFilter home, inside the form where to choose options for displayed filters. This way, the match on the pattern is directly executed in a database query. In my scratch, I got to the point where a af_pattern = xxx check is performed. However, TablePager uses the function select(), where according to the manual you can only build queries with equality and IS NULL. Since we actually need a query with RLIKE, I was wondering if there's another way to achieve this, other than rewriting the select function itself.

ADD: Since this limitation is only valid for arrays, another solution would be to join all conditions into a single string within AF, adding REGEXP where necessary. I would prefer a more elegant and efficient way out, though.

regexp may depend on rdbms, so this sounds not like a possible way.

You can use LIKE with Database::buildLike and Database::anyString

@Umherirrender Thanks for the answer! In the meanwhile, I simply added a call to IDatabase:makeList to join standard conditions and manually added an af_pattern REGEXP xxx if needed. It actually seems to work, or at least it did for all of the simple tests I performed. Now I can also think about adding a radio button for switching between REGEXP and LIKE.

Daimona claimed this task.Feb 16 2018, 11:37 AM

I still need some time to refine the patch, but claiming since it's between scratch and final code.

Daimona added a comment.EditedFeb 16 2018, 2:58 PM

I'm now facing another trouble: to avoid excessively expensive queries with regexps, it would be a nice idea to cap the execution time. The only available way I could find is to use max_execution_time, which however doesn't seem to work. Does someone has an explanation for this or another solution? Thanks.

ADD: I can't find any simple and efficient solution. If I still won't be able to find one before tomorrow, I'll probably rewrite the filtering part, overriding TablePager::formatRow to display the row only if there's a match, i.e. filtering the query result directly in PHP.

Change 411588 had a related patch set uploaded (by Daimona Eaytoy; owner: Daimona Eaytoy):
[mediawiki/extensions/AbuseFilter@master] [WIP] Add search for filter patterns

https://gerrit.wikimedia.org/r/411588

Change 411593 had a related patch set uploaded (by Daimona Eaytoy; owner: Daimona Eaytoy):
[mediawiki/extensions/AbuseFilter@master] [WIP] Add search for filter patterns

https://gerrit.wikimedia.org/r/411593

Daimona added a comment.EditedFeb 22 2018, 8:50 PM

We are at a good point with the patch, but there's one particular thing to be discussed yet: we need to display the string matched by the query. My first proposal was to add a column in Special:AbuseFilter with the pattern matching the query. It was capped at 50 characters centered on the match, showing instead the first 50 characters of the filter by default. @Huji proposed instead a text snippet to be added under the row with the filter, supposedly again with the match highlighted and maybe showing a bit more characters. These are the first ideas, and we need some opinions on which to implement. Of course, any other idea might be proposed as well!

ADD: In the first case, we should also decide whether to show the portion of the pattern by default or only when the user inserted a search pattern. The second option will probably show it only if something was searched.

Adding Matěj as reviewer of the patch.

Huji added a comment.Mar 5 2018, 2:40 AM

@Daimona the layout of the added fields is not optimal. Here is how it looks like if you make the window half-screen (not full screen, but only 50% the width of a standard monitor).

A better idea would be this: have one row for Search within rules; have a second row labeled "Search mode", with the three radio buttons.

Huji added a comment.Mar 5 2018, 2:46 AM

Here is another problem. Imagine a filter with this rule:

added_lines rlike '(this|that|other|foo|bar|lorem|ipsum|dolor|sit|amet|consectetur|adipiscing |elit)'

(And we have much much longer rules in use on many WMF projects). If I search for "lorem" here, I get a very long result in the Patterns box, so much so that it causes a horizontal scrollbar:

Here is a zoomed-in view:

I strongly believe an extra column on the right is *not* the way to go.

What about making the whole table horizontally scrollable? Something like overflow: auto;?

@Huji The search field is easy to fix as you proposed, which would indeed be much better. But for the extra column, I don't see many alternatives, since we need to stick to TablePager methods. We need to keep the snippet inside the existent table: adding a new, dedicated row would break the column layout, while in developing vertically we would need to add the snippet inside an existing cell, which would be as obtrusive as it is now. A scrollable table would probably be easier and better, wouldn't it?

Huji added a comment.Mar 5 2018, 3:59 PM

A scrollable table would be good.

Daimona added a comment.EditedMar 5 2018, 4:40 PM

Implemented scrollable table via overriding table class + CSS. There might be an ever quicker solution, this seems pretty fair and acceptable though.

EDIT: Sounds like jenkins is broken, console output doesn't show the error. Could you please take a look?

Huji closed this task as Resolved.Mar 30 2018, 1:20 AM
Huji removed a project: Patch-For-Review.
Huji added a project: User-notice.
Huji moved this task from To Triage to Announce in next Tech/News on the User-notice board.

Change 411593 merged by jenkins-bot:
[mediawiki/extensions/AbuseFilter@master] Add search for filter patterns

https://gerrit.wikimedia.org/r/411593