I have reported this previously here: https://www.mediawiki.org/wiki/Topic:Wuw1yi6lcck2fujy
I am trying to get rendered HTML for a bunch of pages. I am pretty sure I am limiting myself to 200 requests per second to page/html API endpoint, and I have User-Agent set, but after few minutes I am starting to get 429 responses. Have I misunderstood something about the rate limit for that API endpoint? I am making those 200 requests in parallel though.
I tested with 100 requests per second as well. I managed to make 45852 requests in 8 minutes and after that they started failing with 429. This is a bit less than 100 requests per second.
Example response (for when using 200 requests per second limit on my side):
If you report this error to the Wikimedia System Administrators, please include the details below. Request from IP via cp3056 cp3056, Varnish XID 528119086 Upstream caches: cp3056 int Error: 429, Too Many Requests at Wed, 04 May 2022 16:48:01 GMT
I am sad that expected rate limit from hyperswitch does not get reported: https://github.com/wikimedia/hyperswitch/blob/2d4480149ae4d1969eec83063ddec05c9021574d/lib/filters/ratelimit_route.js#L32-L37