Page MenuHomePhabricator

Create a user right that allows ignoring the spam blacklist
Open, Stalled, MediumPublic

Description

Since this list is (ab)used more and more on Meta, disrupting the work of Commons administrators tagging images as copyright violations (source) and it is very inflexible so URLs like http://www.google.de/url? (\bgoogle\..*?\/url\?) are blacklisted, there is the need that at least established users can override it.

AbuseFilter and the Title blacklist are much more flexible in this area.

Thanks in advance.


Version: unspecified
Severity: normal
See Also:
T57794: Allow file sources to be exempted from spam blacklist

Details

Reference
bz34928
Related Gerrit Patches:
mediawiki/extensions/SpamBlacklist : masterAdd `sboverride` right to override the spam blacklist
mediawiki/extensions/SpamBlacklist : masterAdd user right to bypass blacklist

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
bzimport raised the priority of this task from to Medium.Nov 22 2014, 12:09 AM
bzimport set Reference to bz34928.
bzimport added a subscriber: Unknown Object (MLST).
Rillke created this task.Mar 3 2012, 11:39 AM

Turelio001 wrote:

I can confirm every word of the requester's statement; these kind of spamfilterung is simply an unnecessary and unwelcome disruption of the daily admin work.

That may introduce new problems, since only some people could add some content.

· Admin adds {{copyvio|...}}
· Uploader removes the tag.
· Normal user tries to revert.

Or more subtle:
· Admin talks about some site in the VP, not realising it is blacklisted.
· Bot tries to archive it, but can't add the url to the archive page.

(In reply to comment #2)
A warning to the admin adding such links should be sufficient.

This needs fixing asap, and really shouldn't take very long at all.

This really shouldn't be implemented per comment 2.

(In reply to comment #5)

This really shouldn't be implemented per comment 2.

Then you're just making life impractical for Admins. As a matter of fact, both issues described in comment 2 can easily be overcome with a new 'spam-filter bypass' userright which should be added to the bot, rollbacker, and admin usergroups. I think that anyone with these bits is trustworthy enough to not deliberately introduce spam links.

(In reply to comment #4)

This needs fixing asap, and really shouldn't take very long at all.

Please see http://www.mediawiki.org/wiki/Bugzilla/Fields for the meaning of the severity and priority values. Resetting.

Agreed with comment 2. Wikis with bad spam blacklists should sort those out rather than making their sysops unaware of the issues with it.

(In reply to comment #8)

Agreed with comment 2. Wikis with bad spam blacklists should sort those out

There are no "bad" entries in the spam blacklist. But as soon as it happens that the url matches a regex in the meta-blacklist ([[:m:Spam blacklist]]), searching begins why this action was prevented. Then, at least providing a meaningful error message with
a) The origin
b) The regexp matching
c) How to fix that (e.g. local whitelist or removal from global blacklist)
is required.

rather than making their sysops unaware of the issues with it.

They are not "unaware" if a warning as suggested under comment 3 is added. API also supports warnings.

Please consider that this extension is bundled with mediawiki and thus being used by thousands of wikis, a good number of them very small.
Admins of those wikis should have the option to not be bothered by the spam filter.

Restricted Application added a subscriber: Luke081515. · View Herald TranscriptJan 24 2016, 3:56 PM
Base added a subscriber: Base.Nov 2 2016, 12:36 PM
Bugreporter reopened this task as Open.Jul 7 2017, 11:01 AM
Bugreporter added a subscriber: Bugreporter.

Per duplicated bugs and an non-WMF request. I think we can first create the user right but grant it to nobody, then discuss how can this right be used.

That may introduce new problems, since only some people could add some content.
· Admin adds {{copyvio|...}}
· Uploader removes the tag.
· Normal user tries to revert.

This is T17450.

Or more subtle:
· Admin talks about some site in the VP, not realising it is blacklisted.
· Bot tries to archive it, but can't add the url to the archive page.

Probably we can add the right by default to bots. Currently several bots are already affected as they can remove sections including blacklisted links (which are added before they are blacklisted) but can not add them to archive pages. This results in lost of discussion sections which is not easy to discover.

If someone with this right triggers the spam blacklist, please:

  • Warn the user first (similar to an AbuseFilter warning)
  • Tag the edit so other users are aware that the edit matched the blacklist but has overridden it, for transparency and comment T36928#383614

Then we need another right to suppress the warning, and grant it to bots.

zhuyifei1999 added a comment.EditedJul 7 2017, 11:17 AM

@zhuyifei1999: See here and here (in Chinese).

That suggests that adding an entry to spamblacklist should warn users about existing links to the blacklisted site.

This is T17450.
Probably we can add the right by default to bots. Currently several bots are already affected as they can remove sections including blacklisted links (which are added before they are blacklisted) but can not add them to archive pages. This results in lost of discussion sections which is not easy to discover.

That's playing whack-a-mole. eg. What if the user does not have rollback permissions and tries to revert via 'undo'?

The purpose of SpamBlacklist is to prevent the wiki from being spammed, not to prevent a particular links from being added. I think a normal admin will not spam the wiki on purpose.

That's playing whack-a-mole. eg. What if the user does not have rollback permissions and tries to revert via 'undo'?

They can just report the page to a venue, e.g. using {{editprotected}}. We already meet this problem in Abusefilter.

An admin won't spam the wiki on purpose, but can cause edit disruption (if not warned) or abuse the right preventing other users from editing, specially if to mitigate the problem we're going to add the right to a bunch of less trusted groups like rollback.

They can just report the page to a venue, e.g. using {{editprotected}}. We already meet this problem in Abusefilter.

Which would be needless if:

  • after blacklist addition, the link cannot be added in any way, and
  • before blacklist addition, the link can be discovered and removed.

Besides, more user right = more complexity in "user right hierarchy", making it harder for newbies to learn to improve our projects.
(Someone already complained about how complex our rules are)

MBH added a comment.Jul 7 2017, 12:31 PM

Bot edits should be excluded from spam blacklisting. "Currently several bots are already affected as they can remove sections including blacklisted links (which are added before they are blacklisted) but can not add them to archive pages. This results in lost of discussion sections which is not easy to discover" - this is common, old and very disruptive problem with ruwiki forum archive bot.

MBH added a comment.Jul 15 2017, 2:28 PM

Besides, more user right = more complexity in "user right hierarchy", making it harder for newbies to learn to improve our projects

It's an obvious idea that users, who have big flags, have some special rights, that ordinary users doesn't have. In ruwiki, many users (even some admins) thinks that admins can overcome spamfilter.

Kf8 added a subscriber: Kf8.Jul 15 2017, 11:11 PM
waldyrious added a subscriber: waldyrious.
Krd added a subscriber: Krd.Apr 23 2018, 3:48 AM
DannyS712 added a subscriber: DannyS712.

This is honestly technically simple. If no user groups should be given the right by default, thats fine.

Restricted Application added a project: User-DannyS712. · View Herald TranscriptNov 23 2019, 2:45 AM

Change 552618 had a related patch set uploaded (by DannyS712; owner: DannyS712):
[mediawiki/extensions/SpamBlacklist@master] Add sboverride right to override tho spam blacklist

https://gerrit.wikimedia.org/r/552618

The patch provided:

  • Adds a new user right, sboverride[1]
  • That, by default, is not given to anyone
  • That allows the user to override the spam blacklist
  • On both normal edits and file uploads

[1] Name inspired by tboverride allowing users to bypass title blacklist

I don't think it unreasonable that sysops should have it by default, and maybe bots.

As a bot operator this patch would make a big difference to save dead links and fix usurped domains.

I prefer being blocked in normal operations, it allows to see where a problem exists and fix it. But there are cases, such as replacing a usurped domain with archive URL versions (from before usurpation), the filter makes it impossible. It can impact thousands at a time.

I don't think it unreasonable that sysops should have it by default, and maybe bots.

I agree, but configuration can be tweaked later - my first priority was getting the right created, so that sites can use it by manually granting it to admins.

@Cyberpower678 as far as WMF defaults go, may want to use this like the 'flood' flag. Create a standalone group for the permission, allow sysops to +/- the group, and perhaps bots to +self/-self from it.

@Anomie gave a -2 on gerrit, saying "The problem is concern over the impact when only some users are prevented and others are not." - this patch being accepted would have absolutely no impact on some users being prevented and others not, since it doesn't give the right to anyone.

  • For non-WMF wikis, this conversation doesn't need to be had, since it should be up to wiki operators to decide
  • For WMF wikis, this conversation should be had, but until it is had, the feature should be available to non-WMF wikis

By not assigning the right to anyone in the initial patch, I had hoped to avoid exactly this issue of requiring the discussion before implementing the technical framework

I don't understand the issue. We have abusefilters already that block additions of specific content by users matching specific criteria, and noone is complaining there. How is this different, albeit a bit more generalized?

I don't understand the issue. We have abusefilters already that block additions of specific content by users matching specific criteria, and noone is complaining there. How is this different, albeit a bit more generalized?

The issue is that abuse filters can be tuned to specifically block a certain user, group of users, simply to log, warn, or deny the specific action. A spam blacklist outright blocks an edit that introduces anything matching a pattern in the regex list. It can't be overridden and it can't be targeted towards specific users.

There may be the problem. An user that owns the right adds a text containing a spam link, but the user doesn't know the link is in the spam list. I think, there should be an alert message while saving a text with adding links contained in the spam list.

There may be the problem. An user that owns the right adds a text containing a spam link, but the user doesn't know the link is in the spam list. I think, there should be an alert message while saving a text with adding links contained in the spam list.

There probably should be, but perfection is the enemy of good. Its better to give this functionality and then make it pretty

DannyS712 changed the task status from Open to Stalled.EditedDec 10 2019, 9:00 PM

Patch is ready, but has -2 pending approval

AnnAngela added a subscriber: AnnAngela.EditedThu, Mar 12, 1:12 PM

IDK why the right is needless, I have a bot running a task to fix pages by null editing whole page in a non-WMF wiki, and there are always pages cannot be edited because they included spam links come from meta-wiki blacklist.

And I have no choice but manually add the domain in local whitelist :(

I really appreciate if bot can have right to override it.

Fae added a comment.Thu, Mar 12, 1:24 PM

Come on. This has been discussed for literally seven years. "Perfection is the enemy of the good" indeed.

Change 584712 had a related patch set uploaded (by Dbarratt; owner: Dbarratt):
[mediawiki/extensions/SpamBlacklist@master] Add user right to bypass blacklist

https://gerrit.wikimedia.org/r/584712

Whoops... created a duplicate of this task (as well as a duplicate patch). I noticed a discussion on Wikidata about this topic.

Change 584712 abandoned by Dbarratt:
Add user right to bypass blacklist

Reason:
Duplicate of I516dc68ec7a2dfaa82647feb67ec9bd264b8c380

https://gerrit.wikimedia.org/r/584712