Page MenuHomePhabricator

remove noratelimit from bot group for Wikidata
Closed, ResolvedPublic

Description

Problem:
Currently, bots are exempt from rate limits on Wikidata. This is needed for some tools like MassMessage and Nuke. However "normal" bots should have some sane limit on edit speed because otherwise the infrastructure suffers.

Proposed solution:

  • We remove noratelimit from the bots group and add accounts that require noratelimit to "accountcreator" group

Acceptance criteria:

  • the Bots group on Wikidata is rate-limited
  • MassMessage (User:MediaWiki message delivery) and Nuke are not limited and in the "accountcreator" group
  • announcement has been made before the change is deployed to production

Probable breakdown? (now potentially outdated):

  • Write config patch
  • CHECK, write and deploy code for i18n messages if needed in WikimediaMessages extension
  • Config changes should probably be reviewed by Adam, Amir or Lucas?
  • Coordinate with ComCom for a deployment slot
  • Deploy

Useful links:

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 19 2020, 5:33 PM
Lydia_Pintscher triaged this task as High priority.Jul 19 2020, 5:34 PM
Bugreporter added a subscriber: Bugreporter.EditedJul 20 2020, 3:40 AM

I instead think we should:

  • Remove noratelimit from all bots
  • Add accounts that require noratelimit to "accountcreator" group

Before this we should:

  • Find all bots that edits more than 90/minute
  • Communiate to them

Before this we should:

  • Find all bots that edits more than 90/minute
  • Communiate to them

I am already monitoring user edit rates occasionally with a custom Python script that reads the Wikimedia RC stream, and I have contacted several bot operators regarding their edit rates when I saw excessively high values. I wasn't particularly successful, I have to admit.

Unfortunately the situation with maxlag>5 for considerable fractions of a day due to the WDQS bottleneck has brought us to a situation where there is sort of a competition for as many editing resources as possible among bot operators. As long as there is neither a policy enforcing a limit nor a hard technical limit, the greediest operator will simply win this competition. There are already quite some bot operators who challenge the Wikidata bot policy by editing up to maxlag=6 or even 7, as they probably think that nobody will notice.

We also have to remember that exact throttling to a targeted edit rate is not simple at all. Some tools such as QuickStatements do not allow to control the edit rate at all and they just hammer as many edits as possible to the servers, particularly for users that are not rate limited; quite some bot operators simply use a framework such as Pywikibot and might not be overly proficient when it comes to throttling (although pywikibot in particular does not create that many issues I think); and finally, it is quite difficult to monitor the own edit rate at all, and to realize that one causes issues.

I'm afraid I have to say that I do not see a chance for a fair distribution of our limited edit resources among users and bot operators as long as we do not have a hard technical edit rate limit.

Regarding the proposed solution:

  • "flooders" should be treated the same as bots
  • I would like to see a way to limit all other unlimited users' edit rates as well (sysops, apparently global rollbackers, ...). When I use my sysop account with QuickStatements and run a single batch, it can easily go up to 400 or 500 edits per minute and there is no possibility for me to slow down. Not fair for all users without such elevated rights.

It needs a thorough assessment which functionality categorically needs to be unlimited (mass message, nuke?, ...). Potential users of these functionality should be able to temporarily add no-ratelimit to their accounts *for the purpose of using these functions only*, and should remove it thereafter.

What would happen if a bot reached a rate limit? Would its edits just be delayed, the same as what happens currently when maxlag is reached, or would the edits be dropped/result in an error message? Hopefully it would be the former.

+1 to what @MisterSynergy said.

What would happen if a bot reached a rate limit?

I believe they will be receiving an error message with ratelimited until they have waited enough time. Whether the bot crashes or waits depends on the used framework.

Add accounts that require noratelimit to "accountcreator" group

Quite odd solution.

Addshore updated the task description. (Show Details)Jul 21 2020, 12:16 PM
Addshore updated the task description. (Show Details)Jul 21 2020, 12:20 PM

I instead think we should:

  • Remove noratelimit from all bots
  • Add accounts that require noratelimit to "accountcreator" group

We discussed it just now and unfortunately this will break mass message at least so is not an option :(

Change 618245 had a related patch set uploaded (by Tobias Andersson; owner: Tobias Andersson):
[operations/mediawiki-config@master] DNM WIP: add new limited bot group

https://gerrit.wikimedia.org/r/618245

Lydia_Pintscher added a comment.EditedAug 4 2020, 4:57 PM

ANswers for remaining questions: Let's call it "Slow bots" and announce it before creation. I can write something up. Let's coordinate via chat for a date.

I think the approach is wrong. Let's start with the first assumption. Nuke ( https://www.wikidata.org/wiki/Special:Nuke ) runs under your own account. Used it plenty of times. As an admin I have noratelimit so having noratelimit on a completely different group is not relevant. MassMessage is just an extension that runs in the background. It is a bot. It shouldn't go too fast and it shouldn't crash on ratelimits. Relevant historic bugs: T192690 and T184948 .

Just add a sane (relatively high) ratelimit to the current bots group or keep it as is. The current proposed solution is to try to work around the problem.

Restricted Application added a subscriber: pywikibot-bugs-list. · View Herald TranscriptAug 6 2020, 2:23 PM

When it is created and current bots moved to new group it would be a breaking change.

Just add a sane (relatively high) ratelimit to the current bots group or keep it as is. The current proposed solution is to try to work around the problem.

Well, that was exactly what we did in T184948: limit page creation and edit rate on Wikidata and you disagreed with it (T184948#4145480)

Just add a sane (relatively high) ratelimit to the current bots group or keep it as is. The current proposed solution is to try to work around the problem.

Well, that was exactly what we did in T184948: limit page creation and edit rate on Wikidata and you disagreed with it (T184948#4145480)

No, that's not what you did. You added a ratelimit to a group which had the "noratelimit" right assigned to it causing all sorts of breakage.

Help me understand your proposal better. Sorry if I miss anything obvious. Bots and sysops have "noratelimit" right attached to their user rights groups. We can either have another bots right group that doesn't have the "noratelimit" right or apply a forced ratelimit to the current bot group (which has noratelimit right attached to it). The former is this ticket, the latter is the other. What is the third option here? Stripping bots group out of its noratelimit right?

When it is created and current bots moved to new group it would be a breaking change.

https://www.wikidata.org/wiki/Wikidata:Stable_Interface_Policy#Notification_Policy

Changes to non-stable interfaces may not be announced, even if they are breaking changes.

And these changes does not fall under any stable interface.

WMDE-leszek updated the task description. (Show Details)Sep 3 2020, 1:50 PM

Awaiting announcement

I instead think we should:

  • Remove noratelimit from all bots
  • Add accounts that require noratelimit to "accountcreator" group

I do strongly agree with this, as this would avoid the burden of re-assigning bots from a group to a group, and also i feel this is systematicaly better solution.

Before this we should:

  • Find all bots that edits more than 90/minute
  • Communiate to them

+1

Instead of a new user group, a better solution would be to remove, for the Wikidata bot group, the noratelimit permission and adjust in rOMWC wmf-config/InitialiseSettings.php an acceptable rate limit for all bots if needed. Currently non-autoconfirmed users gets 8 edits per minute, while autoconfirmed gets 90 edits per minute. 90 edits/minute. Should be an acceptable limit, and just requires a simple config patch instead of (a) create a new group and (b) move accounts between groups.

Ok so here's where we're at:

  • asking bots to be nice: This has happened. It's playing whack-a-mole. Admins who care are rightfully upset that this is not a workable solution and we've had complaints asking to not make this a whack-a-mole game anymore. It's also not fair to the people who are actually trying to be good citizens and abite by the rules just to then be outrun by people who don't care or know what they're doing. This is not an acceptable state for me.
  • just removing noratelimit from the existing bot group: that's been done in the past and caused bad side-effects with extensions that actually require it like MassMessage and Nuke (?).
  • adding the new bot group: I think it'd have been an elegant solution as it gives control to the admins on-wiki but apparently not :D

So that leaves the suggestion to remove noratelimit from all bots and add accounts that require noratelimit to "accountcreator" group. @Ladsgroup: Would this work?

Ok, I see, we only put misbehaving bots in this group.

So that leaves the suggestion to remove noratelimit from all bots and add accounts that require noratelimit to "accountcreator" group. @Ladsgroup: Would this work?

Definitely.

Ok, I see, we only put misbehaving bots in this group.

I think the proposal is the other way around, we add important bots that needs bypassing ratelimit to accountcreator group (like massmessage)

I assume we could define a higher ratelimit for bots in general but not too high.

So that leaves the suggestion to remove noratelimit from all bots and add accounts that require noratelimit to "accountcreator" group. @Ladsgroup: Would this work?

Definitely.

Ok then let's do this. I'll adapt the ticket accordingly.

I still find a bit ridiculous that fast editing bots will be in a group for account creators...

abian added a subscriber: abian.Thu, Oct 1, 5:21 PM

I like this solution. The only right provided by the flag of account creator is noratelimit, and right now no user account has this flag, so we can freely change the group/flag name with MediaWiki:Group-accountcreator and MediaWiki:Group-accountcreator-member to fit its future purpose on Wikidata.

What should this group/flag be called? Superbot? Something more creative? Ideally, the name shouldn't suggest that it's redundant with the bot group/flag, but additional.

Lydia_Pintscher renamed this task from create new rate-limited bot group for Wikidata to remove noratelimit from bot group for Wikidata.Wed, Oct 7, 3:25 PM

What should this group/flag be called? Superbot? Something more creative? Ideally, the name shouldn't suggest that it's redundant with the bot group/flag, but additional.

Ignore rate limits? No rate limit? Fast editors? Fast users? I don't think we should go for superbot, because there are use cases for noratelimit too, such as creating accounts at outreach events.

abian added a comment.EditedThu, Oct 8, 10:39 AM

Ignore rate limits? No rate limit? Fast editors? Fast users? I don't think we should go for superbot, because there are use cases for noratelimit too, such as creating accounts at outreach events.

According to what I've just checked, in the history of Wikidata the accountcreator flag has been granted four times, and on none of those occasions has it been used for anything, the users didn't create any account with it. If we anticipate that such a need may arise in the future, we could introduce the event coordinator group, but for now that need doesn't seem to exist.

I would prefer it to be understood that belonging to this group is an undesirable exception, and I wouldn't like users who don't belong to it (almost everyone) to be considered handicapped or punished because they're categorised as "slow" or "limited", as opposed to "fast" users; in fact, all users can continue to edit at a high rate. Other names as mediocre as "superbot" come to mind, from more serious to more irreverent: structural user/account, greedy user/account, exhausting/draining user/account, speed(ing) offender, freeloader, racehorse (racing goat? 🤨)…

Per announcement, unless there is a strong opposition from the community to this change we will implement it on October 20th.
Thanks a lot to everyone giving constructive feedback under this ticket!

About MassMessage and Nuke: MassMessage edits are apparently made by a special system account, so adding that to the Account creators group should solve the MassMessage problem. And Nuke acts through the user who issued the command, but the command is limited to Administrators and that group already has the noratelimit right, so they should not be affected.

However, the Wikidata staff group doesn’t have the right to add the MassMessage system user to the Account creators group (and neither do regular Administrators like Amir or Marius). If I understand correctly, only Bureaucrats are allowed to add users to the Account creators group, and we only have three of those on Wikidata (list of Bureaucrats; plus probably some global group members?). @Lydia_Pintscher: should we use a maintenance script to add the system user to the group, or ask a Bureaucrat to do it? The maintenance script would be simpler, but if I recall correctly, that leaves no trace in the user rights log, so it would be less transparent on-wiki.

Change 634938 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[operations/mediawiki-config@master] Remove noratelimit from Wikidata bot group

https://gerrit.wikimedia.org/r/634938

Actually, @hoo might be able to add User:MediaWiki message delivery to Account creators, as a steward? I’m not sure – global groups confuse me.

Yeah if hoo can't do it then let's ask ymblanter.

Change 634938 merged by jenkins-bot:
[operations/mediawiki-config@master] Remove noratelimit from Wikidata bot group

https://gerrit.wikimedia.org/r/634938

Mentioned in SAL (#wikimedia-operations) [2020-10-20T11:09:56Z] <lucaswerkmeister-wmde@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:634938|Remove noratelimit from Wikidata bot group (T258354)]] (duration: 00m 56s)

Change deployed. So far, the edit rate at Wikidata Edits hasn’t come crashing down, which is probably good.

Actually, @hoo might be able to add User:MediaWiki message delivery to Account creators, as a steward? I’m not sure – global groups confuse me.

Note: Stewards can technically do literally anything (ie. anything the web interface allows for) – so yes, @hoo would be technically able to do the change. However, stewards will never use their authority on projects with locals who can do the change - in this case, Wikidata's bureaucrats.

I would support using createAndPromote.php, since this is in fact a sysadmin decision, and not something that can be decided about by any community.

I'm however not sure why is this necessary: AFAICS, https://github.com/wikimedia/mediawiki-extensions-MassMessage/blob/master/includes/Job/MassMessageServerSideJob.php#L68 doesn't actually call User::pingLimiter, so it doesn't seem to be affected?

Well, IIRC we’ve had problems with MassMessage being rate-limited in the past, so it’s probably better to be safe than sorry and still add it to the group. I’ve left a message on Ymblanter’s talk page.

Well, IIRC we’ve had problems with MassMessage being rate-limited in the past, so it’s probably better to be safe than sorry and still add it to the group. I’ve left a message on Ymblanter’s talk page.

I think the usual place to post such requests is https://www.wikidata.org/wiki/Wikidata:Bureaucrats%27_noticeboard - but that's mostly Ymblanter anyway!

Change 618245 abandoned by Tobias Andersson:
[operations/mediawiki-config@master] Add new slow-bot group for Wikidata

Reason:
https://phabricator.wikimedia.org/T258354#6509213

https://gerrit.wikimedia.org/r/618245