comms.http.request catches any retry-after and stores its value with the Throttle object of the current site. But this is only used with Throttle.lag() which is called by Request.submit if the Action API has a maxlag code. Recently the Wikimedia frontend restricts API calls and gives a 429 http code with retry-after value. This is not respected yet but a wait cycle is started and the wait cycle delay is mostly greater than the given retry-after value. Anyway the retry-after should be respected with the next API access.
Description
Description
Details
Details
Related Changes in Gerrit:
| Subject | Author | Repo | Branch | Lines +/- | |
|---|---|---|---|---|---|
| get_delay: consider retry_after in delay calculation | Shivaansh Singh | pywikibot/core | master | +15 -7 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | BUG REPORT | Xqt | T414170 All Github actions tests fails due to 429 status code | ||
| Resolved | BUG REPORT | Xqt | T414354 retry-after is not respected except there is a server lag reported via API |
Event Timeline
Comment Actions
Change #1225619 had a related patch set uploaded (by Shivaansh Singh; author: Shivaansh Singh):
[pywikibot/core@master] Subject - Fix HTTP 429 handling to respect Retry-After header (T414354)
Comment Actions
Change #1225619 had a related patch set uploaded (by Shivaansh Singh; author: Shivaansh Singh):
[pywikibot/core@master] Add setuptools dependency to fix test failures
Comment Actions
Implementation idea:
The retry-after value is already read from response header and stored in throttle.retry_after attribute of the current site.
Throttle.get_delay():
- Calculate current_delay as the maximum of
- writedelay and retry_after value, or
- delay and retry_after value
- If retry_after is taken into account, clear it.
- That's it? Yes that should be sufficient in most cases.
Advanced features:
- Instead of using Throttle.process_multiplicity which counts the parallel processes of the current site, take all parallel processes instead.
- increase config.putthrottle if 429 error was triggered by a write action and config.minthrottle for read requests.
Comment Actions
Change #1225619 merged by jenkins-bot:
[pywikibot/core@master] get_delay: consider retry_after in delay calculation