Page MenuHomePhabricator

429 "too many requests" while requesting "what transcludes page" for many templates in AWB
Closed, ResolvedPublicBUG REPORT

Description

I have a list of 922 templates in "make list" textfield. I did ctrl-a, right click - add selected to list from - what transcludes page (all NS) and got this error: (429) Too many requests (f061ab2). I'm logged in, have bot and admin flag and loaded NoLimits plugin. I did something similar many times before and such error didn't occured.

Event Timeline

This is probably going to need some behaviour changes on the AWB side, and a followup release...

Starting today, I've been getting the same error from

  1. Creating a list from a Category (recurse user defined) with a recursion depth of 2
  2. Doing a regular typo run on a list of a few hundred articles (it stops with the error every so often if it's going through the list quickly)

It seems like a quite a bit we use AWB for is getting restricted. Or is this just a temporary maintenance situation?

I guess there's some bug in AutoWikiBrowser so that those requests lack authentication cookies; otherwise you wouldn't get that error, which can only happen for logged-out users.

Having said that, while I agree with @Reedy that AWB might need to change behaviour, I'll try to solve the issue temporarily.

I have added an hotfix that should go live soon-ish (in the next hour or so).

If in two hours you still see the blocks, please let me know on this task.

Joe claimed this task.

I'll tag the task as solved, please let us know if you still enocounter problems. Thanks to @Reedy for the assistance over the weekend!

Already mentioned to @Reedy: the current official release should quietly retry any HTTP error after a delay that starts at 5 seconds. and escalates. But it does not honor a RetryAfter header (there is code in the tree to fix that). Also I haven't looked carefully enough at the bot mode path, and/or it could be coming from a custom HTTP submission in that plugin.

Opened T416505 for investigation.

@Joe: what did the hotfix do? I'd like to take that into account.

Wikimedia was updated some time last year to return error 429 (Too Many Requests) when someone is using resources too quickly, and the error is accompanied by an advisory pause time (Retry-After header). I've received in it some of my own client-side code, only ever getting a pause time of 1 second. The cure is to wait that period and retry the same call. Success isn't guaranteed though; the pause request is re-evaluated.

I implemented that in the main read-next-article code (although it's not been officially released) but the make-a-list code was still vulnerable. I've a fix for that, but need to be able to force the condition to test it!

Retry-After can also come with a 503 Service Unavailable, but I haven't seen it.

Fixed on the client side with AWB SVN 13014 (honor Retry-After headers in list generation). This is not tested live however; maybe the hotfix by @Joe dealt with the issue on the server side. Still, it's the right thing to do.