Page MenuHomePhabricator

Enforce upload rate limits for bots on commons
Open, MediumPublic

Description

While investigating T248151: Big number of uploads from DPLA bot it has become obvious that bots on commons are not currently subject to upload rate limits.

For both performance, reliability and capacity planning reasons I think we should be enforcing (generous, but not unbounded) rate limits on file uploads, hence this task. Such limits will make it easier to be within capacity planning plans and protect systems from being overloaded, but not hinder regular upload activities.

Event Timeline

Change 582046 had a related patch set uploaded (by Reedy; owner: Reedy):
[operations/mediawiki-config@master] Enforce upload ratelimits on commons

https://gerrit.wikimedia.org/r/582046

Bots have noratelimit rights. With this right there is no way to have the upload action ratelimited, but edit actions not.

https://commons.wikimedia.org/wiki/Special:ListGroupRights

Bots have noratelimit rights. With this right there is no way to have the upload action ratelimited, but edit actions not.

https://commons.wikimedia.org/wiki/Special:ListGroupRights

Yes there is. That's what &can-bypass on $wgRateLimits is for. Though it will affect all groups, not just bots

Bots have noratelimit rights. With this right there is no way to have the upload action ratelimited, but edit actions not.

https://commons.wikimedia.org/wiki/Special:ListGroupRights

Reedy is working with can-bypass feature of ratelimit - https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/313941/

That seems okay.

Volans triaged this task as Medium priority.Mar 24 2020, 9:40 AM

Change 588988 had a related patch set uploaded (by Cparle; owner: Cparle):
[operations/mediawiki-config@master] Enable quality constraints on production commons

https://gerrit.wikimedia.org/r/588988

Change 582046 abandoned by Reedy:

[operations/mediawiki-config@master] Enforce upload ratelimits on commons

Reason:

https://gerrit.wikimedia.org/r/582046