Nakavo - Rate Limiting Query
Closed, InvalidPublic
Actions

Assigned To

None

Authored By

	NakavoDev
	Apr 9 2026, 7:14 PM

Project Tags

Referenced Files

	F75436999: image.png
	Apr 9 2026, 7:58 PM

Subscribers

Description

Hello Team,We’re (Rami and Ionut) a small two-person company (Nakavo) building a trip planning app that uses the Wikidata and Wikimedia APIs to retrieve images for OSM-based locations.
Our user profile: https://commons.wikimedia.org/wiki/User:NakavoDev
Setup:
∙ Two API keys under the same username (one for dev, one for prod)
∙ REST endpoints (w/rest.php) for metadata, and https://upload.wikimedia.org URIs for image downloads
∙ All requests include an Authorization header (Bearer token) and a User-Agent (Nakavo/Dev and Nakavo/Prod)
∙ Our background worker distributes requests evenly over time, designed to stay within the 5,000 requests/hour limit
Despite this, we’re receiving 429 responses before reaching what we believe the limit to be.
Could you help us understand:

What might be causing the 429 errors given our current request distribution?
Is the 5,000/hour limit per API key or shared across all keys under the same account?

Happy to provide logs or additional details if helpful.
Thank you

Related Objects

Mentioned Here: P18 my paste!
P948 (An Untitled Masterwork)
P2176 serving hack
P3451 Q21558717 CPU profile?
P4291 PHPUnit with both HHVM and PHP5
P5775 (An Untitled Masterwork)
P6802 (An Untitled Masterwork)
P8592 3035-reimage-fail

Event Timeline

NakavoDev created this task.Apr 9 2026, 7:14 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 9 2026, 7:14 PM

Reedy added a project: SRE.Apr 9 2026, 7:15 PM

Reedy added a project: Traffic.

Two API keys under the same username (one for dev, one for prod)

Hi, which exact type of "API keys" is this about?

and https://upload.wikimedia.org URIs for image downloads

How exactly do you construct such image URIs?

stay within the 5,000 requests/hour limit

From which documentation does the "5000" number come from?

Also see https://www.mediawiki.org/wiki/Wikimedia_APIs/Rate_limits for general information and recent changes - thanks!

In addition to what @Aklapper has mentioned above, please also include the full error message (that you see along with the 429). Thanks.

Hey,

Hi, which exact type of "API keys" is this about?

We've created the api keys in the app management portal

How exactly do you construct such image URIs?

We do this in 3 steps:

We get the statements for an entity i.e. https://www.wikidata.org/w/rest.php/wikibase/v1/entities/items/Q84/statements. From here we extract the file names belonging to one of the following: P2176, P18, P948, P3451, P8592, P4291, P4640, P5775, P6802, P8517, P5775
For the selected images we call the commons rest api to get the file info : https://commons.wikimedia.org/w/rest.php/v1/file/London Montage 2016.png
We extract one of the links returned (the smallest one > 800 defaulting to the first one)

From which documentation does the "5000" number come from?

I can't find the documentation around this, it's possible that my information was incorrect. What would be the correct rate?

Extra Info:

We haven't reached pass ~150 requests overall for the prod account as we have only started testing. We do have a worker that it's scheduling these calls using a time-linear distribution of 2400 per hour (We can reduce this number if based on your recommendation)

The headers we are using:
- Authorization: Bearer {Key}
- User-Agent: Nakavo/Dev (will switch to Nakavo/Prod soon) -> we can keep this simply Nakavo if you prefer.
- Api-User-Agent: Same as above (I've read that in some cases it's recommended to also pass this)

Captured Exception Message

"<!DOCTYPE html><html lang=en><meta charset=utf-8><title>Wikimedia Error</title><style>* { margin: 0; padding: 0; }body { background: #fff; font: 15px/1.6 sans-serif; color: #333; }.content { margin: 7% auto 0; padding: 2em 1em 1em; max-width: 640px; display: flex; flex-direction: row; flex-wrap: wrap; }.footer { clear: both; margin-top: 14%; border-top: 1px solid #e5e5e5; background: #f9f9f9; padding: 2em 0; font-size: 0.8em; text-align: center; }img { margin: 0 2em 2em 0; }a img { border: 0; }h1 { margin-top: 1em; font-size: 1.2em; }.content-text { flex: 1; }p { margin: 0.7em 0 1em 0; }a { color: #0645ad; text-decoration: none; }a:hover { text-decoration: underline; }code { font-family: sans-serif; }summary { font-weight: bold; cursor: pointer; }details[open] { background: #970302; color: #dfdedd; }.text-muted { color: #777; }@media (prefers-color-scheme: dark) {  a { color: #9e9eff; }  body { background: transparent; color: #ddd; }  .footer { border-top: 1px solid #444; background: #060606; }  #logo { filter: invert(1) hue-rotate(180deg); }  .text-muted { color: #888; }}</style><meta name=color-scheme content=light dark><div class=content role=main><a href=https://www.wikimedia.org><img id=logo src=https://www.wikimedia.org/static/images/wmf-logo.png srcset=https://www.wikimedia.org/static/images/wmf-logo-2x.png 2x alt=Wikimedia width=135 height=101></a><div class=content-text><h1>Error</h1><p>Too many requests - please contact noc@wikimedia.org to discuss a less disruptive approach or instead use thumbnail images in sizes listed on https://w.wiki/GHai. (0c640b1)</p></div></div><div class=footer><p>If you report this error to the Wikimedia System Administrators, please include the details below.</p><p class=text-muted><code>Request served via cp3078 cp3078, Varnish XID 38078995<br>Upstream caches: cp3078 int<br>Error: 429, Too many requests - please contact noc@wikimedia.org to discuss a less disruptive approach or instead use thumbnail images in sizes listed on https://w.wiki/GHai. (0c640b1) at Thu, 09 Apr 2026 08:40:19 GMT<br><details><summary>Sensitive client information</summary>IP address: 104.248.175.184</details></code></p></div></html>"

By you extract one of the links... What do you mean?

Are you always getting thumbs? Or are you sometimes (often?) requesting the originals based on size?

In T422872#11806792, @Reedy wrote:

By you extract one of the links... What do you mean?

Are you always getting thumbs? Or are you sometimes (often?) requesting the originals based on size?

Hey Reedy,

To download one image we call the commons rest api to get the file info (i.e. https://commons.wikimedia.org/w/rest.php/v1/file/London Montage 2016.png). From here we check the following:

preferred.url
original.url
thumbnail.url

Out of these 3 we select the first image having a width >= 800 (in this example it would be thumbnail.url). If no such size exist (meaning all are having a width < 800) we select the largest one.

Would this align or contradict the preferred way of downloading images using commons media? We only care about having a good enough size for displaying it using either a mobile / tablet device.

Do you know what stats of thumbnail vs original you’re requesting?

Generally, thumbnails are definitely preferred, so if you’re preferring original because it’s first match, that will start to explain your issues.

And you can actually request certain thumbnail sizes even if not presented to you via that API call.

Some of them will end up being generated, but many should be cached, especially for standard sizes if you’re picking used images.

In T422872#11808793, @Reedy wrote:

Do you know what stats of thumbnail vs original you’re requesting?

Generally, thumbnails are definitely preferred, so if you’re preferring original because it’s first match, that will start to explain your issues.

And you can actually request certain thumbnail sizes even if not presented to you via that API call.

Some of them will end up being generated, but many should be cached, especially for standard sizes if you’re picking used images.

I am not a comparison stat comparing the difference in files.

I will change the logic so that we always default to a thumbnail in this case, thank you for sharing. I will monitor the changes and hopefully no more 429 will show up

For this example, would thumbnail be considered the url under the thumbnail object? I noticed it's missing the thumb path

{
    "title": "London Montage 2016.png",
    "file_description_url": "//commons.wikimedia.org/wiki/File:London_Montage_2016.png",
    "latest": {
        "timestamp": "2016-07-01T20:50:03Z",
        "user": {
            "id": 2892925,
            "name": "AlexTref871"
        }
    },
    "preferred": {
        "mediatype": "BITMAP",
        "size": null,
        "width": 428,
        "height": 599,
        "duration": null,
        "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/5/51/London_Montage_2016.png/500px-London_Montage_2016.png"
    },
    "original": {
        "mediatype": "BITMAP",
        "size": 2245752,
        "width": 1000,
        "height": 1400,
        "duration": null,
        "url": "https://upload.wikimedia.org/wikipedia/commons/5/51/London_Montage_2016.png"
    },
    "thumbnail": {
        "mediatype": "BITMAP",
        "size": 2245752,
        "width": 1000,
        "height": 1400,
        "duration": null,
        "url": "https://upload.wikimedia.org/wikipedia/commons/5/51/London_Montage_2016.png"
    }
}

I am thinking in
-> use the preferred.url if the size is > 800
-> use 960 and manually construct the CDN thumb uri otherwise (960 would be an accepted value here https://www.mediawiki.org/wiki/Common_thumbnail_sizes)

I will change the logic so that we always default to a thumbnail in this case, thank you for sharing. I will monitor the changes and hopefully no more 429 will show up

@NakavoDev Hi, is there still an issue, or can this ticket be resolved?

Hey,
This ticket can be closed now. We don't have too much traffic at the moment and we haven't noticed this issue again. We can ask for your guidance again in the future should we face the same issue again. Thank you all

Thanks for the quick reply! Closing per last comment.

Nakavo - Rate Limiting QueryClosed, InvalidPublicActions

Description

Related Objects

Event Timeline

Nakavo - Rate Limiting Query
Closed, InvalidPublic
Actions