Page MenuHomePhabricator

Write RFC about site-wide edit rate limiting
Closed, ResolvedPublic

Description

Basically turning this to a RFC:

I have an idea. I think we should use PoolCounter (which is basically a SaaS, Semaphore as a service) to put a cap on edits happening on wikidata at the same time. This is being used when an article is being reparsed as well, so not too many mw nodes parse an article at the same time (The Michael Jackson effect).

Basically once a request realizes it's going to make an edit in Wikidata, it decreases the semaphore of "edit cap on Wikidata" (let's say initialized by value of 10, meaning only ten edits at the same time can happen in Wikidata). Once the semaphore reaches zero, PoolCounter keeps the 11th mw node trying to lock waiting and responds once one of the ten current ones finishes, if it's more let's say twenty, it just responds with "Too many edits happening". This means edit saving time might be artificially slow when there are more ten edits happening at the same time. Not that this already works fine with parsing articles (look at the blog post), I used this a while back on ores to prevent more than four IPs requesting ores at the same to avoid intentional and unintentional DoSes, It works fine as well.

PoolCounter is a pretty reliable service with almost zero down time and already have a good support inside mediawiki.