Page MenuHomePhabricator

Images randomly fail to load (Error 429)
Closed, ResolvedPublicBUG REPORT

Description

Images on both Wikipedia and Wiktionary randomly fail to load. By "randomly" I mean it changes on each page reload or browser restart and can happen on any page with images. I couldn't find any pattern or a way to reproduce it.
This has only started happening recently.

OS: Windows
Browser: Tor, Firefox, Brave
(cookies and cache are always cleared on exit)

Console logs:

NS_BINDING_ABORTED
Status 429
The stream mediawiki.client.session_tick is out of sample. No event will be sent
MediaWikiMetricsClientLogger.js:19:9
Status code: 429
Too many requests (76af2b0)
The stream mediawiki.client.session_tick is out of sample. No event will be sent
MediaWikiMetricsClientLogger.js:19:8
Failed to load resource: the server responded with a status of 429 ()
The stream mediawiki.web_ui_actions is out of sample. No event will be sent
MediaWikiMetricsClientLogger.js:19
The stream mediawiki.client.session_tick is out of sample. No event will be sent
MediaWikiMetricsClientLogger.js:19

Steps to replicate the issue (include links if applicable):

  • Go to any Wikipedia or Wiktionary page with images
  • See that some of the images are broken

What happens?:
Some of the images are broken.

What should have happened instead?:
All of the images displaying correctly.

Software version (on Special:Version page; skip for WMF-hosted wikis like Wikipedia):
N/A

Other information (browser name/version, screenshots, etc.):
Provided above.

Event Timeline

Aklapper changed the task status from Open to Stalled.Feb 25 2026, 10:37 AM

Hi @BrokenImages1234, thanks for taking the time to report this! Unfortunately this Wikimedia Phabricator task lacks some information.
If you have time and can still reproduce the situation: Please add a more complete description to this task. That should be

  • a clear and complete list of exact steps to reproduce the situation, step by step, so that nobody needs to guess or interpret how you performed each step,
  • what happens after performing these steps to reproduce,
  • what you expected to happen instead,
  • a full link to a web address where the issue can be seen,
  • the web browser(s) and web browser version(s) that you tested.

You can edit the task description by clicking Edit Task. Ideally, a good description should allow any other person to follow these steps (without having to interpret steps) and see the same results. Problems that others can reproduce can get fixed faster. Thanks again!

No idea how any of this helps, but I edited it. The issue is non-reproducible, it occurs randomly.

Browser: Tor, Firefox, Brave

Please provide web browser version(s) that you tested.

See that some of the images are broken

What is shown in the Network tab of your browser's developer tools in this case?

Please provide web browser version(s) that you tested.

Have just tested it again in Tor v15.0.7. Hard-refreshed the page three times in a row, and it was always different images that failed to load each time.

What is shown in the Network tab of your browser's developer tools in this case?

"NS_BINDING_ABORTED".

TorNetwork.png (1,900×168 px, 14 KB)

Bugreporter closed this task as Invalid.EditedMar 2 2026, 7:47 AM
Bugreporter subscribed.

Many countries and many people has slow or intermittent Internet access, either because of low bandwidth or Internet censorship. So it is somewhat normal to connect to Wikimedia server. WMF can give little help on that. So close as invalid unless there is evidence of this being a problem affecting multiple regions. Advices:

  • If you can access Google or Facebook without issue (meaning you have at least basic Internet access, try to find a usable proxy or VPN (but not Tor) to access Wikipedia.
  • Add <IP> upload.wikimedia.org to your hosts file where <IP> is one of 208.80.154.240, 208.80.153.240, 198.35.26.112, 185.15.59.240, 185.15.58.240, 103.102.166.240 or 195.200.68.240, and see whether you can see images normally.

This has only started happening about a few weeks ago. Doesn't matter if I use Tor, a VPN or nothing at all, so this is not a specific connection issue.
Tor bridges in particular work the same for everyone.

Try opening this page in Tor with bridges enabled, for example:
https://en.wikipedia.org/wiki/Sushi
Some images fail to load with a "429" error.

Oh, if it is returning 429, then please provide a screenshot with more detail as following:

  • Click the red number "429" in browser console
  • Click "Response" at the right of newly-shown detail field
  • Scroll to bottem
  • Post a screenshot here.

Or alternatively:

  • Right-click and open the 429 image at a new tab
  • If an error message is displayed, then post a screenshot here.

I can see this behaviour on the (internal) OfficeWiki, on the Contact List, on Firefox 148.0 on Linux (Ubuntu snap packaging), from a connection with the Init7 internet provider in Switzerland.

The 429s look like this:

image.png (584×542 px, 94 KB)

Loading the image in a new tab does work, so does asking the browser to reload the image in place (but it disappears again if I reload the page).

Can you add a screenshot of the body (not just header) of the response?

GET https://upload.wikimedia.org/wikipedia/commons/thumb/4/49/Kaiten-zushi_005.jpg/500px-Kaiten-zushi_005.jpg
NS_BINDING_ABORTED

Status 429
Version HTTP/2
Transferred 939 B (0 B size)
Referrer Policy strict-origin-when-cross-origin
Request Priority Low
DNS Resolution System

HTTP/2 429 
date: Tue, 03 Mar 2026 03:33:18 GMT
server: Varnish
x-cache: cp3080 int
x-cache-status: int-front
server-timing: cache;desc="int-front", host;desc="cp3080"
strict-transport-security: max-age=106384710; includeSubDomains; preload
report-to: { "group": "wm_nel", "max_age": 604800, "endpoints": [{ "url": "https://intake-logging.wikimedia.org/v1/events?stream=w3c.reportingapi.network_error&schema_uri=/w3c/reportingapi/network_error/1.0.0" }] }
nel: { "report_to": "wm_nel", "max_age": 604800, "failure_fraction": 0.05, "success_fraction": 0.0}
x-client-ip: 2a0e:97c0:3e3:460:1337:b01b:1337:2
access-control-allow-origin: *
access-control-expose-headers: Age, Date, Content-Length, Content-Range, X-Content-Duration, X-Cache
timing-allow-origin: *
retry-after: 1
content-type: text/html; charset=utf-8
content-length: 1987
x-request-id: 500d8573-c564-49e1-99ce-6e415c196377
x-analytics: 
X-Firefox-Spdy: h2

The "Response" tab only shows "No response data available for this request" for some reason... Am I doing something wrong?

The whole thing, bar a couple of redacted elements (for another image on officewiki):

GET
	
scheme
	https
host
	upload.wikimedia.org
filename
	/wikipedia/commons/thumb/8/86/WMF_homepage_banner_image_01.png/90px-WMF_homepage_banner_image_01.png
Address
	[2a02:ec80:600:ed1a::2:b]:443
Status
429
VersionHTTP/2
Transferred940 B (0 B size)
Referrer Policyno-referrer
Request PriorityLow
DNS ResolutionSystem

    	
    access-control-allow-origin
    	*
    access-control-expose-headers
    	Age, Date, Content-Length, Content-Range, X-Content-Duration, X-Cache
    content-length
    	2166
    content-type
    	text/html; charset=utf-8
    date
    	Tue, 03 Mar 2026 08:49:28 GMT
    nel
    	{ "report_to": "wm_nel", "max_age": 604800, "failure_fraction": 0.05, "success_fraction": 0.0}
    report-to
    	{ "group": "wm_nel", "max_age": 604800, "endpoints": [{ "url": "https://intake-logging.wikimedia.org/v1/events?stream=w3c.reportingapi.network_error&schema_uri=/w3c/reportingapi/network_error/1.0.0" }] }
    retry-after
    	10
    server
    	Varnish
    server-timing
    	cache;desc="int-front", host;desc="cp6008"
    strict-transport-security
    	max-age=106384710; includeSubDomains; preload
    timing-allow-origin
    	*
    x-analytics
    	
    x-cache
    	cp6008 int
    x-cache-status
    	int-front
    x-client-ip
    	[redacted]
    X-Firefox-Spdy
    	h2
    x-request-id
    	[redacted]
    	
    Accept
    	image/avif,image/webp,image/png,image/svg+xml,image/*;q=0.8,*/*;q=0.5
    Accept-Encoding
    	gzip, deflate, br, zstd
    Accept-Language
    	en-US,en;q=0.9
    Cache-Control
    	no-cache
    Connection
    	keep-alive
    Cookie
       [redacted] 
    Host
    	upload.wikimedia.org
    Pragma
    	no-cache
    Priority
    	u=5, i
    Sec-Fetch-Dest
    	image
    Sec-Fetch-Mode
    	no-cors
    Sec-Fetch-Site
    	same-site
    User-Agent
    	Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:148.0) Gecko/20100101 Firefox/148.0

The whole thing, bar a couple of redacted elements (for another image on officewiki):

That is a different thing (non-standard thumb size on private wikis) which I‌ fixed today noon. Can you purge the page and see it's fixed or not?

Oops, sorry for hijacking the thread, then (want me to put that somewhere else?)

It seems to have fixed the main page (which is the one I was getting that from this morning), but not the contact list, for which I still have a bunch of 429s after purge, with no obvious difference that what i see)

GET
	https://upload.wikimedia.org/wikipedia/commons/thumb/9/92/Isabelle_Hurbain-Palatin.jpg/250px-Isabelle_Hurbain-Palatin.jpg
Status
429
VersionHTTP/2
Transferred939 B (0 B size)
Referrer Policyno-referrer
Request PriorityLow
DNS ResolutionSystem

    	
    access-control-allow-origin
    	*
    access-control-expose-headers
    	Age, Date, Content-Length, Content-Range, X-Content-Duration, X-Cache
    content-length
    	1986
    content-type
    	text/html; charset=utf-8
    date
    	Tue, 03 Mar 2026 16:49:50 GMT
    nel
    	{ "report_to": "wm_nel", "max_age": 604800, "failure_fraction": 0.05, "success_fraction": 0.0}
    report-to
    	{ "group": "wm_nel", "max_age": 604800, "endpoints": [{ "url": "https://intake-logging.wikimedia.org/v1/events?stream=w3c.reportingapi.network_error&schema_uri=/w3c/reportingapi/network_error/1.0.0" }] }
    retry-after
    	1
    server
    	Varnish
    server-timing
    	cache;desc="int-front", host;desc="cp6008"
    strict-transport-security
    	max-age=106384710; includeSubDomains; preload
    timing-allow-origin
    	*
    x-analytics
    	
    x-cache
    	cp6008 int
    x-cache-status
    	int-front
    x-client-ip
    	[redacted]
    X-Firefox-Spdy
    	h2
    x-request-id
    	[redacted]
    	
    Accept
    	image/avif,image/webp,image/png,image/svg+xml,image/*;q=0.8,*/*;q=0.5
    Accept-Encoding
    	gzip, deflate, br, zstd
    Accept-Language
    	en-US,en;q=0.9
    Cache-Control
    	no-cache
    Connection
    	keep-alive
    Cookie
    	WMF-Uniq=[redacted]; GeoIP=[redacted]
    Host
    	upload.wikimedia.org
    Pragma
    	no-cache
    Priority
    	u=4, i
    Sec-Fetch-Dest
    	image
    Sec-Fetch-Mode
    	no-cors
    Sec-Fetch-Site
    	same-site
    TE
    	trailers
    User-Agent
    	Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:148.0) Gecko/20100101 Firefox/148.0

If it's now a standard size then it's another rate limit and I don't know what's going on with those :( sorry. I think the Traffic team will investigate.

@BrokenImages1234, @ihurbain: Hi. I can confirm I can reproduce the issue as well. I have a possible theory I would like to confirm and then tomorrow we can work on hopefully resolving this issue.

Do you have third-party cookies disabled by default? Can you try enabling them just for this test and then check if the images load? Because that works for me and I think I know but I have asked for the input of more smarter people than me in the meantime.

@ssingh
Yes, of course. I don't think it's even possible to enable them in Tor...

@ssingh
Yes, of course. I don't think it's even possible to enable them in Tor...

You mentioned Firefox and Brave above. Can you please try with those?

I have third-party cookies disabled and I am apparently also affected by this bug. I'm using Firefox 149 with the "Enhanced Tracking Protection" setting set to "Strict", which blocks (among other things) "Cross-site cookies in all windows". I'm not using any weird extensions, this is a built-in Firefox setting.

A few of us on the MediaWiki Platform team have discussed the use of cookies on the upload domain with a few people from SRE not so long ago in the context of T414337, and we have talked about how unreliable they're going to be for privacy-conscious users, and increasingly also under the default settings in some browsers.

Thanks for confirming @matmarex. We are discussing this internally and will follow up.

BrokenImages1234 renamed this task from Images randomly fail to load to Images randomly fail to load (Error 429).Mar 5 2026, 5:20 AM
BrokenImages1234 updated the task description. (Show Details)

@ssingh
Have just tried it in Brave and still got the same error:

Failed to load resource: the server responded with a status of 429 ()

At first I thought this was just me using an exotic browser and getting misclassified as a bot, but I'm encountering the same error even in Edge, no third-party cookie blocking as far as I'm aware. E.g. on https://en.wikipedia.org/wiki/Time_zone, which through the country icons has a shedload of images.

Change #1248443 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/puppet@production] cache::upload: increase limits for non-obvious bots for media files

https://gerrit.wikimedia.org/r/1248443

Copying my comment from https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(technical)#Image_thumbnails_often_failing_to_load: I figured out where to find the data about the number of thumbnail 429s we serve, and added it to a relevant dashboard: https://grafana.wikimedia.org/d/000000034/media. The number of 429s has been increasing over the last 3 weeks or so; note that the lines on this graph have different scales, so it's a steady 40k/sec of HTTP 200 responses, and recently up to 8k/sec of HTTP 429 responses.

image.png (932×296 px, 101 KB)

Note that we are being hammered by scrapers. On non-standard sizes, we are basically issuing 12 429s for each 200 (majority of the 429s are with zero or negative browser score so we are sure they are not legit). It's not easy to distinguish why we issued a 429 on grafana but the increase is not just because of X or Y (for example, yesterday I‌ enabled a different rate limit on non-standard sizes)

I don't know if the chart above can be broken down by size, but this is not just about non-standard sizes. For example https://en.wikipedia.org/wiki/Road_signs_in_Finland needs some 700 HTTP requests to load correctly, and it took me half a dozen refreshes to get past HTTP 429 errors for the thumbnails, although they all use <gallery> defaults of 40px and 120px.

The chart seems to show a somewhat gradual increase, although with two marked steps around February 18 and Mach 1st. Does it mean the caches are being evicted gradually, increasing the chances that requested thumbnails are not in cache? And if so, why? For less visited articles and categories, one could expect thumbnail caches to expire (as bot visits get increasingly blocked, an increasing number of articles will get zero visits in a month or whatever the cache expiration period is), but I would be surprised if standard thumbnail caches expire for an article with 1400 visits/month.

Edit: it's not (only) about caching because I get the same HTTP 429 response even if I hard-refresh the page immediately after I managed to successfully load all thumbnails (and they all seem to be cache hits at that point).

Example response just for the record:

HTTP/2 429 
date: Thu, 05 Mar 2026 13:17:29 GMT
server: Varnish
x-cache: cp3079 int
x-cache-status: int-front
server-timing: cache;desc="int-front", host;desc="cp3079"
strict-transport-security: max-age=106384710; includeSubDomains; preload
report-to: { "group": "wm_nel", "max_age": 604800, "endpoints": [{ "url": "https://intake-logging.wikimedia.org/v1/events?stream=w3c.reportingapi.network_error&schema_uri=/w3c/reportingapi/network_error/1.0.0" }] }
nel: { "report_to": "wm_nel", "max_age": 604800, "failure_fraction": 0.05, "success_fraction": 0.0}
x-client-ip: #redacted
access-control-allow-origin: *
access-control-expose-headers: Age, Date, Content-Length, Content-Range, X-Content-Duration, X-Cache
timing-allow-origin: *
retry-after: 1
content-type: text/html; charset=utf-8
content-length: 1966
x-request-id: 0e2b090f-b2de-4b55-8896-ec98327ec201
x-analytics: 
X-Firefox-Spdy: h2

Change #1248443 merged by Giuseppe Lavagetto:

[operations/puppet@production] cache::upload: increase limits for non-obvious bots for media files

https://gerrit.wikimedia.org/r/1248443

Change #1248487 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/puppet@production] varnish: raise limit for upload also for cache misses

https://gerrit.wikimedia.org/r/1248487

Change #1248487 merged by Ssingh:

[operations/puppet@production] varnish: raise limit for upload also for cache misses

https://gerrit.wikimedia.org/r/1248487

I don't get rate-limited anymore in Tor and Brave but still do in Firefox. Is it because I use an old version of it?

Many factors come into play; one of them is using old browser versions. Please update if possible.

@Aklapper
I would, but they removed the feature that was the reason I switched to Firefox in the first place.
Anyway, spoofing my user-agent does solve the issue, but... Why would it be an issue in the first place? Why would any bots ever use outdated browser versions...?

By now I don't think any further unknown factors come into play for this ticket, so I'd propose to close this ticket.

Why would any bots ever use outdated browser versions...?

We are lucky they are so stupid.

matmarex claimed this task.

I think the known issues are resolved. If you experience a similar problem, please file a new task.