Page MenuHomePhabricator

retry-after value retrieved from http header looks wrong
Open, HighPublic

Description

With T144023 we introduced using 'retry-after' value from http response_headers.

But sometimes the maxlag value is very high compared to the retry_after value which might to short:
see https://api.travis-ci.org/v3/job/458291459/log.txt for example (T210322)

See also https://www.mediawiki.org/w/index.php?title=Manual:Maxlag_parameter&oldid=85948
Refer T172293 too.

Event Timeline

Xqt triaged this task as High priority.Nov 28 2018, 1:57 PM

Change 476274 had a related patch set uploaded (by Xqt; owner: Xqt):
[pywikibot/core@master] [bugfix] Increase the delay if maxlag >> retry-after

https://gerrit.wikimedia.org/r/476274

Getting a better estimate for Retry-After is T172293. This doesn't seem to really be a bug other than that existing feature request.

My path above is a proposal to use a portion of maxlag parameter as a prediction for the next lag if retry_after is far away from maxlag.

retry_after maxlagwait time
5105
5306
55010
510020
525050
51000120
55000120

Change 476274 merged by jenkins-bot:
[pywikibot/core@master] [bugfix] Increase the delay if maxlag >> retry-after

https://gerrit.wikimedia.org/r/476274

Change #1354211 had a related patch set uploaded (by Xqt; author: Xqt):

[pywikibot/core@master] maxlag: Prefer maxlag over Retry-After for throttling delays#

https://gerrit.wikimedia.org/r/1354211

The Retry-After value is always 5 seconds; there is a proposal in T172293 to change this. The recommendation in Manual:Maxlag parameter is to wait for the number of seconds specified in the maxlag error response. With the patch given above the wait cycle changes to this:

retry_after maxlagold wait timenew wait time
510510
530630
5501050
510020100
525050120
51000120120
55000120120