Page MenuHomePhabricator

Increase the rate limit for moves on Commons for file movers to 100/minute
Closed, DeclinedPublic

Description

https://commons.wikimedia.org/wiki/Commons_talk:File_renaming#Ratelimit_exceeded

File mover Arjoopy runs into the rate limit here after 16 legitimate moves. Commons is rather different from other wikis, we may move tons of file pages in a short period.

As I don't know exactly what rate limit is tied to, the most neat solution would be to increase the move limit for filemovers or, if that's not possible, increase the move limit for autopatrollers, patrollers and image reviewers. (file movers are typically in one of those groups as well)

As for the number, I'd say increase it to 100 per minute or so. File mover/autopatrol/patrol/image reviewer are all only assigned by humans, so only trusted users get those rights.

Edit: I understand the rate limit for moves can be adjusted for just file movers.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 17 2019, 5:38 PM
AlexisJazz updated the task description. (Show Details)Jul 17 2019, 5:43 PM
DannyS712 added a subscriber: DannyS712.

@AlexisJazz is there any consensus or local discussion for this, either about increasing the ratelimit in general, what user groups it should apply to, or what the new ratelimit should be?

Restricted Application added a project: User-DannyS712. · View Herald TranscriptJul 17 2019, 5:43 PM
AlexisJazz added a comment.EditedJul 17 2019, 5:59 PM

@DannyS712 no, currently not. But the rate limit has one purpose: to curb vandalism without limiting any ordinary use. Limiting vetted users to the degree that it hampers their work means it's not configured correctly.

Increasing the move limit for anyone who can't move files isn't needed. There is little need to discuss which user groups: it should be file mover, or if that's not technically possible, autopatroller+patroller+license reviewer. (patroller and license reviewer are supersets of autopatroller) Page move vandalism by those vetted groups is extremely rare to non-existent.

And it should be raised to the point where legitimate users don't run into it. That's not quite about consensus, this is a technical thing. We never voted to have this move rate limit in the first place.

@Bawolff maybe you can make some stats?

DannyS712 added a comment.EditedJul 17 2019, 6:01 PM

The current rate limits for commons are

'+commonswiki' => [ // T132930
	'edit' => [
		'newbie' => [ 8 * 15, 60 * 15 ], // T225148
		'user' => [ 900, 60 * 3 ], // T194864
		// Higher rate limit for trusted users
		'image-reviewer' => [ 10500, 60 * 3 ],
		'patroller' => [ 10500, 60 * 3 ],
		'autopatrolled' => [ 10500, 60 * 3 ],
	],
	'upload' => [
		// 380 uploads per 72 minutes
		'user' => [ 380, 4320 ],
		// Effectively no upload rate limit for members of these groups
		'image-reviewer' => [ 999, 1 ],
		'patroller' => [ 999, 1 ],
		'autopatrolled' => [ 999, 1 ],
	],
],

Assuming there are no objections, I'll upload a patch to allow users with filemover to move 100 pages per minute

@DannyS712 that's on top of the MediaWiki defaults? MediaWiki default is 8 moves per minute AFAIK, which would seem plausible if the system was counting Arjoopy's moves over 2 minutes.

If Commons has no rate limit for moves it's a bug, because Arjoopy came nowhere near 10500 edits.

@DannyS712 that's on top of the MediaWiki defaults? MediaWiki default is 8 moves per minute AFAIK, which would seem plausible if the system was counting Arjoopy's moves over 2 minutes.
If Commons has no rate limit for moves it's a bug, because Arjoopy came nowhere near 10500 edits.

Danny listed changes of defaults. This rate limits as evaluated by Commons:

[urbanecm@mwmaint1002 ~]$ mwrepl commonswiki
Welcome to HipHop Debugger!
Type "help" or "?" for a complete list of commands.

Note: no server specified, debugging local scripts only.
If you want to connect to a server, launch with "-h" or use:
  [m]achine [c]onnect <servername>

hphpd> var_dump($wgRateLimits['move']);
var_dump($wgRateLimits['move']);
array(2) {
  ["newbie"]=>
  array(2) {
    [0]=>
    int(2)
    [1]=>
    int(120)
  }
  ["user"]=>
  array(2) {
    [0]=>
    int(8)
    [1]=>
    int(60)
  }
}

hphpd> ^Dquit
[urbanecm@mwmaint1002 ~]$

I don't have a problem with changing this to the number you propose, but I'd appreciate if this is at least announced to the community.

And it should be raised to the point where legitimate users don't run into it. That's not quite about consensus, this is a technical thing. We never voted to have this move rate limit in the first place.

Well, you also never voted to have a rate limit for failing to solve a CAPTCHA. Community decides about changes of defaults, while devs choose some sane defaults usable for most of the wikis. BTW, I don't think a formal voting is necessary (unless mandated by Commons's policies). Just posting a note somewhere visible (preferable a Village pump) and waiting for possible objections should be just fine. We just don't want to get blamed for "doing something against community's will", not to block you from doing work.

@Urbanecm Thanks, I understand it now, the configuration is on top of the defaults.

I made an announcement at https://commons.wikimedia.org/wiki/Commons:Village_pump#Increasing_the_rate_limit_for_page_moves.

Thank you, @AlexisJazz. Let's wait for a week or so and process this unless someone opposes.

AlexisJazz updated the task description. (Show Details)Jul 17 2019, 6:48 PM

@DannyS712 no, currently not. But the rate limit has one purpose: to curb vandalism without limiting any ordinary use. Limiting vetted users to the degree that it hampers their work means it's not configured correctly.

This is not entirely true. The main purpose is to maintain the stability of the servers, with the (very welcome but) secondary purpose of inhibiting vandalism until a sysop can block them. That means that well-meaning community members may from time to time bump into one of these limits.

We never voted to have this move rate limit in the first place.

[OT] Measures to have stable infrastructure is not a voting / popularity contest. :)
Also see https://www.mediawiki.org/wiki/Bug_management/Development_prioritization#Why_wasn't_I_consulted_about_these_changes%3F

DannyS712 triaged this task as Medium priority.Jul 17 2019, 7:23 PM
AlexisJazz added a comment.EditedJul 17 2019, 7:25 PM

@DannyS712 no, currently not. But the rate limit has one purpose: to curb vandalism without limiting any ordinary use. Limiting vetted users to the degree that it hampers their work means it's not configured correctly.

This is not entirely true. The main purpose is to maintain the stability of the servers, with the (very welcome but) secondary purpose of inhibiting vandalism until a sysop can block them. That means that well-meaning community members may from time to time bump into one of these limits.

T194864#4213365 "The limit was supposed to be an upper bound and not affect actual users. Im sorry for the inconvinence and am totally ok with a higher limit (as long as all groups that you can be auto added to have a reasonable rate limit)" (BWolff)
T194864#4212085 "Well, this was introduced because of vandals editing with speed over 30000 edits per hour (this is too high). " (Urbanecm)

We never voted to have this move rate limit in the first place.

[OT] Measures to have stable infrastructure is not a voting / popularity contest. :)
Also see https://www.mediawiki.org/wiki/Bug_management/Development_prioritization#Why_wasn't_I_consulted_about_these_changes%3F

My point was, if we didn't need a vote to see the introduction of a rate limit, why would a vote be needed to adjust it to make it work as intended? (the way we're doing it now works fine for me)

T194864#4213365 "The limit was supposed to be an upper bound and not affect actual users. Im sorry for the inconvinence and am totally ok with a higher limit (as long as all groups that you can be auto added to have a reasonable rate limit)" (BWolff)

Some context here: Firstly, that's about edit limit, not move limit. This particular rate limit (not all rate limits in general) really was supposed to be an upper bound and not affect actual users. It was a slightly different case. This edit rate limit was firstly introduced as emergency measure just for one wiki because of mass vandalism (as I stated in a comment you quote below), those having access, see T192668 for details. Brian (aka BWollf or bawolff) then introduced a global rate limit of 90/60 for editing, but with the same intention - because the vandal moved elsewhere.

T194864#4212085 "Well, this was introduced because of vandals editing with speed over 30000 edits per hour (this is too high). " (Urbanecm)

This is true, but only for this particular rate limit. See above for details. The move rate limit exists for quite a long time, I think it existed even before the conversation you quote from happened.

We never voted to have this move rate limit in the first place.

[OT] Measures to have stable infrastructure is not a voting / popularity contest. :)
Also see https://www.mediawiki.org/wiki/Bug_management/Development_prioritization#Why_wasn't_I_consulted_about_these_changes%3F

My point was, if we didn't need a vote to see the introduction of noratelimit, why would a vote be needed to adjust it to make it work as intended? (the way we're doing it now works fine for me)

No one ever requested a vote, "just" consensus. Depending on various circumstances, a consensus might be created by a RFC, a formal voting, an informal discussion or - visible announcement with no objections. Anyway, since the current way works for you, I don't think it's necessary to discuss this more, just wanted to explain this little confusion.

4nn1l2 added a subscriber: 4nn1l2.Jul 18 2019, 5:54 AM
SJu added a subscriber: SJu.Jul 18 2019, 4:54 PM

The problem with the rate limit affects not just file moves but also other page moves. E. g. when I rename (manually!) a set of categories in order to harmonize their names with the appropriate category tree, sometimes I have a problem with the rate limit too. I have no problem to wait a moment, but IMHO the limit can be a bit increased. Maybe to twofold?

SJu added a comment.Jul 18 2019, 5:03 PM

Some context here: Firstly, that's about edit limit, not move limit.

Really? I have no problem to move hundreds of files from a category to an other category using Cat-a-lot in a few seconds. But it is a problem to rename about 10 categories in one minute. As I can see, this is affected just by move limit, not by edit limit.

Some context here: Firstly, that's about edit limit, not move limit.

Really? I have no problem to move hundreds of files from a category to an other category using Cat-a-lot in a few seconds. But it is a problem to rename about 10 categories in one minute. As I can see, this is affected just by move limit, not by edit limit.

Please read my comment carefully. The word "that" referred to quotations @AlexisJazz included in their comment. The discussion the quotations come from really is purely about edit limit. On the other hand, this request is purely about move limit.

The problem with the rate limit affects not just file moves but also other page moves. E. g. when I rename (manually!) a set of categories in order to harmonize their names with the appropriate category tree, sometimes I have a problem with the rate limit too. I have no problem to wait a moment, but IMHO the limit can be a bit increased. Maybe to twofold?

As I stated above, there is no technical problem in increasing the rate limit, however, a community request must has community consensus. @AlexisJazz posted a note on https://commons.wikimedia.org/wiki/Commons:Village_pump#Increasing_the_rate_limit_for_page_moves. If there will be no objections within a week or so, this request will be processed.

Some context here: Firstly, that's about edit limit, not move limit.

Really? I have no problem to move hundreds of files from a category to an other category using Cat-a-lot in a few seconds. But it is a problem to rename about 10 categories in one minute. As I can see, this is affected just by move limit, not by edit limit.

"Moving" pages between categories is a bit of a misnomer: it is accomplished by editing those pages, not by actually renaming any page (the category itself may be renamed in the process, of course, but that's a single rename action and doesn't directly impact changing the categorization of its members). Moving pages between categories therefore falls under the edit limit.

Multichill added a subscriber: Multichill.

As pointed out on the village pump, the only person hitting the limit was doing an out of policy rename that shouldn't have happened.

Urbanecm changed the task status from Open to Stalled.Jul 20 2019, 10:58 PM

Thank you. Given this is a separate request, and it is approved by different community process (this seems to be a formal voting, and I'll treat it so), please create another task, to make tracking easier for system administrators. Otherwise, system administrator processing this request might easily oversee that this is two requests in one ticket :-). Thank you in advance.

Note this is an opposed announcement now, which means it can no longer be used as a prove that community consensus was estabilished. As such, please create a voting/RFC/whatever is appropriate according to Commons'policies. This won't be processed before this is done. As such, I'm stalling this and applying Community-consensus-needed. Thank you for your understanding.

Urbanecm renamed this task from Increase the rate limit for moves on Commons to Increase the rate limit for moves on Commons for file movers to 100/minute.Jul 20 2019, 11:01 PM
Urbanecm lowered the priority of this task from Medium to Low.

I don't know if I even support this myself anymore.. The only thing that should be fixed is the error message. @DannyS712, did you already create a task for that?

I don't know if I even support this myself anymore.. The only thing that should be fixed is the error message. @DannyS712, did you already create a task for that?

Excuse me, which error message?

@Urbanecm Arjoopy got an error message about 900 edits per 180 seconds.

@Urbanecm Arjoopy got an error message about 900 edits per 180 seconds.

If that's right, it's quite strange. I've tried that on testwiki, this showed up. This message (actionthrottledtext) is not overriden on Commons, and as far as I can see, there is no other message that mentions edit explicitly. Please create a new task yourself, and copy the error message in verbatim (if you paste a a screenshot and/or a message code, that would be wonderful). This is getting slightly off-topic here.

DannyS712 added a comment.EditedJul 21 2019, 11:04 PM

Noting that the proposal for page moving to have a higher rate limit has been withdrawn.

DannyS712 closed this task as Declined.Aug 7 2019, 7:10 PM

The proposal was archived to https://commons.wikimedia.org/wiki/Commons:Village_pump/Archive/2019/07#Increasing_the_rate_limit_for_page_moves - there were 2 objections to it, and no other support, so no community consensus exists