InstantCommons in various third-party wikis makes a lot of requests to commons.wikimedia.org, about 50k per minute. Even from a single IP, it can be several thousand. We have to make sure that we don't throttle these requests too much.
Idea:
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T399291 Epic: API Rate Limiting Architecture | |||
| Open | None | T412585 Epic: Enforce API rate limits (WE5.1.3c) | |||
| Open | daniel | T398919 Epic: API rate limiting dry run (WE5.1.3b) | |||
| Resolved | None | T412520 Ensure requests originating from InstantCommons on third party wikis doesn't get rate limited too much |
Are they caching image metadata? Maybe it's worth following up with the largest reusers.
Tagging in @KCVelaga_WMF as we discussed this briefly in Lisbon.
I agree with it needing followup with the largest requesters, there may be configuration changes they can effect on their end that would alleviate the load, and we would need confirmation from them that the IP ranges they have are stable anyway.