Hours have passed and I'm still seeing the previous revision on the file page.

The thumbnail links to https://upload.wikimedia.org/wikipedia/commons/3/38/Hana_Vagnerov%C3%A1_v_Show_Jana_Krause_19._5._2021_upout%C3%A1vka_10.png?utm_source=commons.wikimedia.org&utm_campaign=index&utm_content=original which also shows the revision from 26 April!

It actually doesn't seem possible to download the original of the revision I uploaded.

More likely related to T28741: Migrate file tables to a modern layout (image/oldimage; file/filerevision; add primary keys) changes

actually no.. I see the new version just fine. Did you bypass your browser cache ? Because that doesn't get invalidated.

In T425216#11883067, @TheDJ wrote:

actually no.. I see the new version just fine. Did you bypass your browser cache ? Because that doesn't get invalidated.

As I said, I tried a completely separate browser. Now it gets really strange. I made a QR-code of https://upload.wikimedia.org/wikipedia/commons/3/38/Hana_Vagnerov%C3%A1_v_Show_Jana_Krause_19._5._2021_upout%C3%A1vka_10.png?utm_source=commons.wikimedia.org&utm_campaign=index&utm_content=original

On my iPhone, it shows the new revision!
On both of my Android phones, it shows the older 26 April revision.

All are connected to the same WiFi. The iPhone seems to use IPv6 while the Android phones use IPv4 but that shouldn't matter? I normally connect to ESAMS. A traceroute app on my iPhone seems to suggest it's using ESAMS too, but it shows an IPv4 address. Another traceroute app shows an IPv6 address but that app doesn't work.

$ wget "https://upload.wikimedia.org/wikipedia/commons/3/38/Hana_Vagnerov%C3%A1_v_Show_Jana_Krause_19._5._2021_upout%C3%A1vka_10.png?utm_source=commons.wikimedia.org&utm_campaign=index&utm_content=original"
--2026-05-03 01:36:41--  https://upload.wikimedia.org/wikipedia/commons/3/38/Hana_Vagnerov%C3%A1_v_Show_Jana_Krause_19._5._2021_upout%C3%A1vka_10.png?utm_source=commons.wikimedia.org&utm_campaign=index&utm_content=original
Resolving upload.wikimedia.org (upload.wikimedia.org)... 185.15.59.240, 2a02:ec80:300:ed1a::2:b
Connecting to upload.wikimedia.org (upload.wikimedia.org)|185.15.59.240|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6577972 (6.3M) [image/png]
Saving to: ‘Hana_Vagnerová_v_Show_Jana_Krause_19._5._2021_upoutávka_10.png?utm_source=commons.wikimedia.org&utm_campaign=index&utm_content=original’

Hana_Vagnerová_v_Sh 100%[===================>]   6.27M  23.0MB/s    in 0.3s    

2026-05-03 01:36:41 (23.0 MB/s) - ‘Hana_Vagnerová_v_Show_Jana_Krause_19._5._2021_upoutávka_10.png?utm_source=commons.wikimedia.org&utm_campaign=index&utm_content=original’ saved [6577972/6577972]

That's the old 26 April revision again, and wget has no browser cache!

Now I tried enabling a US (New York) VPN.

New revision!!

Disable VPN.

Old revision!!

It just happened again, this time on https://commons.wikimedia.org/wiki/File:Mikros_logo.jpeg.

Old 4.5K revision without VPN:

$ wget "https://upload.wikimedia.org/wikipedia/commons/e/ed/Mikros_logo.jpeg?utm_source=commons.wikimedia.org&utm_campaign=index&utm_content=original"
--2026-05-03 03:17:16--  https://upload.wikimedia.org/wikipedia/commons/e/ed/Mikros_logo.jpeg?utm_source=commons.wikimedia.org&utm_campaign=index&utm_content=original
Resolving upload.wikimedia.org (upload.wikimedia.org)... 185.15.59.240, 2a02:ec80:300:ed1a::2:b
Connecting to upload.wikimedia.org (upload.wikimedia.org)|185.15.59.240|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4636 (4.5K) [image/jpeg]
Saving to: ‘Mikros_logo.jpeg?utm_source=commons.wikimedia.org&utm_campaign=index&utm_content=original’

Mikros_logo.jpeg?utm_sourc 100%[=====================================>]   4.53K  --.-KB/s    in 0s      

2026-05-03 03:17:17 (141 MB/s) - ‘Mikros_logo.jpeg?utm_source=commons.wikimedia.org&utm_campaign=index&utm_content=original’ saved [4636/4636]

New 69K revision through US VPN:

$ wget "https://upload.wikimedia.org/wikipedia/commons/e/ed/Mikros_logo.jpeg?utm_source=commons.wikimedia.org&utm_campaign=index&utm_content=original"
--2026-05-03 03:18:49--  https://upload.wikimedia.org/wikipedia/commons/e/ed/Mikros_logo.jpeg?utm_source=commons.wikimedia.org&utm_campaign=index&utm_content=original
Resolving upload.wikimedia.org (upload.wikimedia.org)... 208.80.154.240, 2620:0:861:ed1a::2:b
Connecting to upload.wikimedia.org (upload.wikimedia.org)|208.80.154.240|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 70763 (69K) [image/jpeg]
Saving to: ‘Mikros_logo.jpeg?utm_source=commons.wikimedia.org&utm_campaign=index&utm_content=original’

Mikros_logo.jpeg?utm_sourc 100%[=====================================>]  69.10K  --.-KB/s    in 0.1s    

2026-05-03 03:18:50 (684 KB/s) - ‘Mikros_logo.jpeg?utm_source=commons.wikimedia.org&utm_campaign=index&utm_content=original’ saved [70763/70763]

https://commons.wikimedia.org/wiki/File:Hana_Vagnerov%C3%A1_v_Show_Jana_Krause_19._5._2021_upout%C3%A1vka_10.png and https://commons.wikimedia.org/wiki/File:Mikros_logo.jpeg now show the new revision for me without VPN. Not sure if someone fixed it or if it fixed itself for these files with time. Also unsure if it can still happen again.

I believe there is a 24 hourly script that checks cross dc consistency or something

In T425216#11887650, @TheDJ wrote:

I believe there is a 24 hourly script that checks cross dc consistency or something

That would explain that.

It's very reproducible actually: https://test.wikipedia.org/wiki/File:T425216_ESAMS_overwrite_test.png

In T425216#11887650, @TheDJ wrote:

I believe there is a 24 hourly script that checks cross dc consistency or something

The rather simpler explanation here is the CDN has a hard limit on the cache expiry, which is 24 hours. So what most likely happened here was that the old copy expired in 24 hours itself.

I don't have a good explanation on why this happened but we have seen it happen in the past, very occasionally. Is this still an issue? If yes, we can try to debug it and see if we spot something.

In T425216#11895597, @ssingh wrote:

In T425216#11887650, @TheDJ wrote:

I believe there is a 24 hourly script that checks cross dc consistency or something

The rather simpler explanation here is the CDN has a hard limit on the cache expiry, which is 24 hours. So what most likely happened here was that the old copy expired in 24 hours itself.

I don't have a good explanation on why this happened but we have seen it happen in the past, very occasionally. Is this still an issue? If yes, we can try to debug it and see if we spot something.

Yeah, no, I was wrong and I misunderstood the problem. I misread that the actual image also differs across the CDN but it clearly does not.

sukhe@cumin1003:~$ sudo cumin "A:cp and not P{cp2041* or cp2042*} and not A:ulsfo" "curl -s https://upload.wikimedia.org/wikipedia/commons/3/38/Hana_Vagnerov%C3%A1_v_Show_Jana_Krause_19._5._2021_upout%C3%A1vka_10.png | md5sum -"
95 hosts will be targeted:
cp[2043-2058].codfw.wmnet,cp[6001-6016].drmrs.wmnet,cp[1100-1115].eqiad.wmnet,cp[5017-5021,5023-5032].eqsin.wmnet,cp[3066-3081].esams.wmnet,cp[7001-7016].magru.wmnet
OK to proceed on 95 hosts? Enter the number of affected hosts to confirm or "q" to quit: 95
===== NODE GROUP =====                                                                                                                
(95) cp[2043-2058].codfw.wmnet,cp[6001-6016].drmrs.wmnet,cp[1100-1115].eqiad.wmnet,cp[5017-5021,5023-5032].eqsin.wmnet,cp[3066-3081].esams.wmnet,cp[7001-7016].magru.wmnet                                                                                                  
----- OUTPUT for command #1: 'curl -s https://...0.png | md5sum -' -----                                                              
2cb40b6504fc47d19c1989fe8af22152  -

So yeah, I am not sure what is happening here (not that I am the best person for this anyway) but this does not seem to be related to the CDN.

In T425216#11896751, @ssingh wrote:

Yeah, no, I was wrong and I misunderstood the problem. I misread that the actual image also differs across the CDN but it clearly does not.

sukhe@cumin1003:~$ sudo cumin "A:cp and not P{cp2041* or cp2042*} and not A:ulsfo" "curl -s https://upload.wikimedia.org/wikipedia/commons/3/38/Hana_Vagnerov%C3%A1_v_Show_Jana_Krause_19._5._2021_upout%C3%A1vka_10.png | md5sum -"
95 hosts will be targeted:
cp[2043-2058].codfw.wmnet,cp[6001-6016].drmrs.wmnet,cp[1100-1115].eqiad.wmnet,cp[5017-5021,5023-5032].eqsin.wmnet,cp[3066-3081].esams.wmnet,cp[7001-7016].magru.wmnet
OK to proceed on 95 hosts? Enter the number of affected hosts to confirm or "q" to quit: 95
===== NODE GROUP =====                                                                                                                
(95) cp[2043-2058].codfw.wmnet,cp[6001-6016].drmrs.wmnet,cp[1100-1115].eqiad.wmnet,cp[5017-5021,5023-5032].eqsin.wmnet,cp[3066-3081].esams.wmnet,cp[7001-7016].magru.wmnet                                                                                                  
----- OUTPUT for command #1: 'curl -s https://...0.png | md5sum -' -----                                                              
2cb40b6504fc47d19c1989fe8af22152  -

So yeah, I am not sure what is happening here (not that I am the best person for this anyway) but this does not seem to be related to the CDN.

?

$ md5sum 3.png
60ac3bdc2b5a68e8cda0f99118117eab 3.png

ESAMS (Amsterdam):
$ openvpn --config Amsterdam.ovpn
$ curl -s https://upload.wikimedia.org/wikipedia/test/4/45/T425216_ESAMS_overwrite_test.png --output - | md5sum -
803be1a3faf69055c88a8e7061170336  -

ULSFO (San Francisco):
$ openvpn --config SanFrancisco.ovpn
$ curl -s https://upload.wikimedia.org/wikipedia/test/4/45/T425216_ESAMS_overwrite_test.png --output - | md5sum -
803be1a3faf69055c88a8e7061170336  -

CODFW (Houston):
$ openvpn --config Texas.ovpn
$ curl -s https://upload.wikimedia.org/wikipedia/test/4/45/T425216_ESAMS_overwrite_test.png --output - | md5sum -
60ac3bdc2b5a68e8cda0f99118117eab  -

EQSIN (Singapore):
$ openvpn --config Singapore.ovpn
$ curl -s https://upload.wikimedia.org/wikipedia/test/4/45/T425216_ESAMS_overwrite_test.png --output - | md5sum -
203dbe0702a1c7ee976c18481893228e  -

DRMRS (Marseille):
$ openvpn --config Marseille.ovpn
$ curl -s https://upload.wikimedia.org/wikipedia/test/4/45/T425216_ESAMS_overwrite_test.png --output - | md5sum -
203dbe0702a1c7ee976c18481893228e  -

MAGRU (São Paulo):
$ openvpn --config SãoPaulo.ovpn
$ curl -s https://upload.wikimedia.org/wikipedia/test/4/45/T425216_ESAMS_overwrite_test.png --output - | md5sum -
203dbe0702a1c7ee976c18481893228e  -

Only Houston got it right.

Another overwrite:

$ md5sum 4.png
64a5c589df8d0d96db7475340b5c4c91  4.png

ESAMS (Amsterdam):
$ openvpn --config Amsterdam.ovpn
$ curl -s https://upload.wikimedia.org/wikipedia/test/4/45/T425216_ESAMS_overwrite_test.png --output - | md5sum -
203dbe0702a1c7ee976c18481893228e  -

ULSFO (San Francisco):
$ openvpn --config SanFrancisco.ovpn
$ curl -s https://upload.wikimedia.org/wikipedia/test/4/45/T425216_ESAMS_overwrite_test.png --output - | md5sum -
203dbe0702a1c7ee976c18481893228e  -

CODFW (Houston):
$ openvpn --config Texas.ovpn
$ curl -s https://upload.wikimedia.org/wikipedia/test/4/45/T425216_ESAMS_overwrite_test.png --output - | md5sum -
60ac3bdc2b5a68e8cda0f99118117eab  -

EQSIN (Singapore):
$ openvpn --config Singapore.ovpn
$ curl -s https://upload.wikimedia.org/wikipedia/test/4/45/T425216_ESAMS_overwrite_test.png --output - | md5sum -
203dbe0702a1c7ee976c18481893228e  -

DRMRS (Marseille):
$ openvpn --config Marseille.ovpn
$ curl -s https://upload.wikimedia.org/wikipedia/test/4/45/T425216_ESAMS_overwrite_test.png --output - | md5sum -
203dbe0702a1c7ee976c18481893228e  -

MAGRU (São Paulo):
$ openvpn --config SãoPaulo.ovpn
$ curl -s https://upload.wikimedia.org/wikipedia/test/4/45/T425216_ESAMS_overwrite_test.png --output - | md5sum -
203dbe0702a1c7ee976c18481893228e  -

Hmm none of them got it? Maybe I should have waited a few minutes after overwriting? *about 20mins later, no new overwrite*

$ md5sum 4.png
64a5c589df8d0d96db7475340b5c4c91  4.png

ESAMS (Amsterdam):
$ openvpn --config Amsterdam.ovpn
$ curl -s https://upload.wikimedia.org/wikipedia/test/4/45/T425216_ESAMS_overwrite_test.png --output - | md5sum -
203dbe0702a1c7ee976c18481893228e  -

ULSFO (San Francisco):
$ openvpn --config SanFrancisco.ovpn
$ curl -s https://upload.wikimedia.org/wikipedia/test/4/45/T425216_ESAMS_overwrite_test.png --output - | md5sum -
203dbe0702a1c7ee976c18481893228e  -

CODFW (Houston):
$ openvpn --config Texas.ovpn
$ curl -s https://upload.wikimedia.org/wikipedia/test/4/45/T425216_ESAMS_overwrite_test.png --output - | md5sum -
60ac3bdc2b5a68e8cda0f99118117eab  -

EQSIN (Singapore):
$ openvpn --config Singapore.ovpn
$ curl -s https://upload.wikimedia.org/wikipedia/test/4/45/T425216_ESAMS_overwrite_test.png --output - | md5sum -
64a5c589df8d0d96db7475340b5c4c91  -

DRMRS (Marseille):
$ openvpn --config Marseille.ovpn
$ curl -s https://upload.wikimedia.org/wikipedia/test/4/45/T425216_ESAMS_overwrite_test.png --output - | md5sum -
203dbe0702a1c7ee976c18481893228e  -

MAGRU (São Paulo):
$ openvpn --config SãoPaulo.ovpn
$ curl -s https://upload.wikimedia.org/wikipedia/test/4/45/T425216_ESAMS_overwrite_test.png --output - | md5sum -
64a5c589df8d0d96db7475340b5c4c91  -

Only Singapore and São Paulo got it right??

sukhe@cumin1003:~$ sudo cumin "A:cp and not P{cp2041* or cp2042*}" "curl -s https://upload.wikimedia.org/wikipedia/test/4/45/T425216_ESAMS_overwrite_test.png | md5sum -"
111 hosts will be targeted:
cp[2043-2058].codfw.wmnet,cp[6001-6016].drmrs.wmnet,cp[1100-1115].eqiad.wmnet,cp[5017-5021,5023-5032].eqsin.wmnet,cp[3066-3081].esams.wmnet,cp[7001-7016].magru.wmnet,cp[4037-4052].ulsfo.wmnet
OK to proceed on 111 hosts? Enter the number of affected hosts to confirm or "q" to quit: 111
===== NODE GROUP =====                                                                                                                
(2) cp[3071,3078].esams.wmnet                                                                                                         
----- OUTPUT for command #1: 'curl -s https://...t.png | md5sum -' -----                                                              
803be1a3faf69055c88a8e7061170336  -                                                                                                   
===== NODE GROUP =====                                                                                                                
(3) cp[2043,2053,2058].codfw.wmnet                                                                                                    
----- OUTPUT for command #1: 'curl -s https://...t.png | md5sum -' -----                                                              
60ac3bdc2b5a68e8cda0f99118117eab  -                                                                                                   
===== NODE GROUP =====                                                                                                                
(38) cp[2048,2050,2055].codfw.wmnet,cp[6001-6016].drmrs.wmnet,cp[1105,1108-1109,1112,1114].eqiad.wmnet,cp[5019,5021,5026,5031].eqsin.wmnet,cp[3066-3069,3072,3076-3077,3080-3081].esams.wmnet,cp7015.magru.wmnet                                                            
----- OUTPUT for command #1: 'curl -s https://...t.png | md5sum -' -----                                                              
203dbe0702a1c7ee976c18481893228e  -                                                                                                   
===== NODE GROUP =====                                                                                                                
(68) cp[2044-2047,2049,2051-2052,2054,2056-2057].codfw.wmnet,cp[1100-1104,1106-1107,1110-1111,1113,1115].eqiad.wmnet,cp[5017-5018,5020,5023-5025,5027-5030,5032].eqsin.wmnet,cp[3070,3073-3075,3079].esams.wmnet,cp[7001-7014,7016].magru.wmnet,cp[4037-4052].ulsfo.wmnet   
----- OUTPUT for command #1: 'curl -s https://...t.png | md5sum -' -----                                                              
64a5c589df8d0d96db7475340b5c4c91  -                                                                                                   
================                                                                                                                      
PASS |████████████████████████████████████████████████████████████████████████████████████| 100% (111/111) [00:08<00:00, 12.77hosts/s]
FAIL |                                                                                              |   0% (0/111) [00:08<?, ?hosts/s]
100.0% (111/111) success ratio (>= 100.0% threshold) for command #1: 'curl -s https://...t.png | md5sum -'.
100.0% (111/111) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.

After purging the CDN cache,

sukhe@deploy1003:~$ echo 'https://upload.wikimedia.org/wikipedia/test/4/45/T425216_ESAMS_overwrite_test.png' | mwscript-k8s --attach -- purgeList.php
⏳ Starting purgeList.php on Kubernetes as job mw-script.eqiad.snudh053 ...
🚀 Job is running.
📜 Attached to stdin/stdout:
Purging 1 urls
Done!

sukhe@cumin1003:~$ sudo cumin "A:cp and not P{cp2041* or cp2042*}" "curl -s https://upload.wikimedia.org/wikipedia/test/4/45/T425216_ESAMS_overwrite_test.png | md5sum -"
111 hosts will be targeted:
cp[2043-2058].codfw.wmnet,cp[6001-6016].drmrs.wmnet,cp[1100-1115].eqiad.wmnet,cp[5017-5021,5023-5032].eqsin.wmnet,cp[3066-3081].esams.wmnet,cp[7001-7016].magru.wmnet,cp[4037-4052].ulsfo.wmnet
OK to proceed on 111 hosts? Enter the number of affected hosts to confirm or "q" to quit: 111 
===== NODE GROUP =====                                                                                                                
(111) cp[2043-2058].codfw.wmnet,cp[6001-6016].drmrs.wmnet,cp[1100-1115].eqiad.wmnet,cp[5017-5021,5023-5032].eqsin.wmnet,cp[3066-3081].esams.wmnet,cp[7001-7016].magru.wmnet,cp[4037-4052].ulsfo.wmnet                                                                       
----- OUTPUT for command #1: 'curl -s https://...t.png | md5sum -' -----                                                              
64a5c589df8d0d96db7475340b5c4c91  -

Why did this happen? This is the same thing I shared before that we have seen it happen in some cases and at least I don't know. Adding smarter people than me (@BBlack) to see if they have some insight.

I just noticed that for https://commons.wikimedia.org/wiki/File:Paul_de_Munnik_(2017).jpg this is messing up MediaViewer for me because the overwrite is a lower resolution (in pixels) than the original (upscaled) file. MV crops the lower part of the image. The difference isn't huge in this particular case so it merely looks odd, but I imagine that when the difference is huge it'll render MV unwatchable.

https://commons.wikimedia.org/w/index.php?title=Commons:Help_desk&diff=prev&oldid=1216270150 was quite possibly also this. As expected, users get confused.

I can still reproduce this. After overwriting https://test.wikipedia.org/wiki/File:T425216_ESAMS_overwrite_test.png with number 8 it still showed 7. So I overwrite with 9 and it's still 7.

$ md5sum *.png
e4ba6e7f4991b1e705b74fa76b9a7f07  0.png
803be1a3faf69055c88a8e7061170336  1.png
203dbe0702a1c7ee976c18481893228e  2.png
60ac3bdc2b5a68e8cda0f99118117eab  3.png
64a5c589df8d0d96db7475340b5c4c91  4.png
6cc09b167d19aec898ea61334d96fb39  5.png
c9eafd857332658bfde730dff187ec81  6.png
2f3f010b9a7fe59ce5e9eeba35e1b602  7.png
5b72809871a47802b93644a9349f86a0  8.png
97466f9bb4f8f5d7ef4bc0f9175cbe4a  9.png

$ curl -s https://upload.wikimedia.org/wikipedia/test/4/45/T425216_ESAMS_overwrite_test.png --output - | md5sum -
2f3f010b9a7fe59ce5e9eeba35e1b602  -

Thanks for reporting @AlexisJazz. We discussed this in Traffic. @BBlack suggested that since it's difficult to debug one-off cases like these, particularly since even the above has resolved by now, we will reproduce by coming up with a unique file name and tracing the request.

The suspicion is that what is happening here is what was briefly discussed above: a PURGE never arrives, or it arrives and completes but we fetch the old file from the app layer and therefore continue serving it, with variance across the cp nodes.

The report date (2 May) aligns well with after the first rollout of this to Commons:

In T414338#11880509, on 1 May 2026:

Change #1269441 merged by jenkins-bot:

[operations/mediawiki-config@master] Enable wgTrackMediaRequestProvenance on Commons

https://gerrit.wikimedia.org/r/1269441

This was reverted on May 15, and then re-rolled earlier this month on 3 Aug 2026:

In T414338#12180587, on 3 Aug 2026:

Change #1319903 merged by jenkins-bot:

[operations/mediawiki-config@master] Re-enable wgTrackMediaRequestProvenance on Commons and Wikipedia

https://gerrit.wikimedia.org/r/1319903

Varnish strips query parameters for GET requests to upload.wikimedia.org URLs. This is presumably why presence, variations, or absence of these doesn't affect caching of thumbnails.

However, perhaps it doesn't strip them on PURGE requests. The equivalent logic on text_cache varnish, e.g. query parameteter sort, does happen both ways for consistency.

Alternatively, MediaWiki could omit them which might address the issue.

Okay, so from WikiFilePage::doPurge > LocalFile::purgeCache we have to main branches:

• LocalFile::purgeThumbnails, which calls File::getThumbUrl which directly returns a URL from FileRepo::getZoneUrl. This is a lower-level method currently unaware of provenance. For thumbnails we apply provenance via transform/modifyClientThumbUrl.
• LocalFile::getUrl, which calls File::getUrl which also directly returns a URL from FileRepo::getZoneUrl, but also applies "original" appendRequestProvenance.

Test plan:

$wgTrackMediaRequestProvenance = true;

http://localhost:4000/index.php?title=File:Jq_mnml_1.jpg&action=purge

[squid] MediaWiki\Deferred\CdnCacheUpdate::purge: /images/thumb/b/bf/Jq_mnml_1.jpg/800px-Jq_mnml_1.jpg /images/thumb/b/bf/Jq_mnml_1.jpg/120px-Jq_mnml_1.jpg /images/b/bf/Jq_mnml_1.jpg?utm_source=localhost&utm_campaign=index&utm_content=original http://localhost:4000/index.php/File:Jq_mnml_1.jpg http://localhost:4000/index.php?title=File:Jq_mnml_1.jpg&action=history

Change #1333533 had a related patch set uploaded (by Krinkle; author: Krinkle):

[operations/puppet@production] varnish: Refactor 19-normalize-thumbnail-url.vtc

https://gerrit.wikimedia.org/r/1333533

In T435283#12248179, @BBlack wrote:

[…] IMHO, the right fix for this is to fix the MW side of this to purge the base URL of the original, without the excess params […]

In T435283#12249761, @Bawolff wrote:

I agree that it seems like mediawiki is as at fault here. I think these utm parameters were added relatively recently as part of T414338. cc'ing @Krinkle

I initially leaned towards fixing the Varnish side instead for reasons I'll discuss. But, since both of you suggest otherwise, I looking at it from the MW side first with the intention to fix Varnish afterwards as well for good measure.

In MediaWiki:

1. I considered adding an argument to File::getUrl to control provenance inclusion in the return value. We'd pass the arg from LocalFile::purgeCache. I decided against this, because this is a low-level base class and stable interface. I'd like to reserve the main positional argument there for something more meaningful/lasting in the future. Plus, its documented purpose couldn't be honored by subclass ForeignAPIFile::getUrl, because the API doesn't support getting a URL without provenance. Now, we don't actually need this from the API, because MediaWiki wouldn't be responsible for purging URLs to files on a remote wiki. We only ever call File::get for purging purposes from LocalFile::purgeCache. But, it shows that File::getUrl is the wrong place to encode this responsibility.
2. I considered adding a new LocalFile::getUrlForPurge method, which would basically duplicate the logic of File::getUrl from the parent class but without the appendRequestProvenance call. This could work, but would leave a different kind of confusion in its trail. File::getUrl is stable to override and it seems weird that a subclass of LocalFile can correctly implement this but then has to somehow discover that LocalFile::getUrlForPurge exists and know to override that as well. This is a bug waiting to happen.
3. I considered a new shared method like getRawUrl and move the stable interface there. This is a larger refactor and breaking change that I'd like to avoid.
4. We could let go of trying to generate the URL correctly bottom-up, and instead duplicate knowledge of the UTM parameters, and strip those from the return value afterwards in LocalFile::purgeCache.

Options 2, 3, or 4 seemed fine with varying compromises however, this last option made me really think about the consequences:

Why should MediaWiki purge a different URL than the canonical URL it uses everywhere in its interfaces?

That would leave MediaWiki purging broken by default on any setup with a cache proxy or CDN in front, unless that CDN is configured the same novel quirk as WMF to strip query params from upload.wikimedia.org on GET but not on PURGE. MediaWiki would promote one link, but then purge a different one.

We apply almost all of our normalizations pre-purge in Varnish VCL. E.g. hostname normalization, URL path normalization, URL query parameter sorting. Why should this be any different?

The main thing i think is weird is the inconsistency of it. I think either choice is fine, but it feels really weird to do it when purging originals but not do it when purging thimbs.

Copy of a message I posted in Wikimedia Commons's Village Pump:

Severe bug in the frontend file cache servers (not serving the effective last version of any original file)

There's a '''severe scheduling bug in the Wikimedia file server cache''' (for the "original source" files), which does NOT reflect the LAST version of ANY updated file, but ALWAYS serves '''some previous''' version BEFORE the last update.

• I think that when a file is updated, the file server cache incorrectly receives an ''update event'' much too early, and then updates itself '''before''' the file has been '''actually''' stored and made available on the file storage server.
• This bug does NOT affect the thumbnail generator which correctly uses the last version and updates the PNG thumbnail correctly, because it receives the update event correctly after the file was actually stored on the file server.
• You can wait for hours or days, there's no way to have the "original source" file made available via its standard (undated) link (e.g. https://upload.wikimedia.org/wikipedia/commons/f/ff/U1CC43.svg, instead of https://upload.wikimedia.org/wikipedia/commons/archive/f/ff/20260902082205%21U1CC43.svg which is supposed to be the link for the version dated 2026-09-02 08:22:05 in the history, but is in fact a link to the version immediately before it!)
• This is NOT specific to this file, in fact I've seen this bug in many other files whose updates were NEVER reflected in their ''original'' versions (i.e. not the generated ''thumbnail'' which are correct), many days (or months!) after their updates.
• Attempts to fix this by uploading the exact same content has no effect (the frontend considers this is a duplicate upload, and discards the upload). If you upload again the "original file" that you've downloaded from the top of the file history, you are in fact uploading an older version!
• Attempts to fix this by editing the file description page has no effect on this bug. Always, the history shows links to the incorrect (non-thumbnail) dated versions.

This means that we can NEVER correctly update the original files (with the exception of their thumbnails), as we are always serving original files one (or several) versions before!

• The '''only workaround for now''' is to upload an additional "nulledit" file (with some "invisible" change, e.g. a non-significant space in SVG files, or in some internal file metadata): this works sometimes, but not always as in the previous example where a nulledit upload was made in an attempt but still used the 2nd version, then a revert which is still served with the 2nd version (but the dated version with the nulledit is correct...). In summary the content of the Wikimedia file cache is completely inconsistant with the effective file versions.
• I found no way to force the frontend file cache servers to download again the effective file version from the backend storage servers (e.g. by appending some ''dummy query string'' such as "?action=purge" to the "upload.wikimedia.org" link); the frontend continues to maintain its own old local version (for extremely long time which is constantly extended as long as the file is demanded because the frontend is never informed that there was an update was available, even days, weeks, or months before).
• The bug also affects the "file preview" (e.g. when you click any file in Commons to display the file in "full screen" mode, it also shows an old version).

I consider this synchronization bug (between the inconstant frontend file cache servers, and the backend storage servers which superficially seem correct, as these latter servers are used by thumbnail generators) being '''severe''' everywhere in Wikimedia for three major reasons:

• It affects ''countless'' files that need to be legitimately updated on Wikimedia Commons, but also files stored in other Wikimedia wikis (e.g. in any localized edition of Wikipedia when these files are not accepted in Commons due to licencing exceptions). The displayed file histories are completely inconsistant! The bug is independant of the file content type (not specific to SVG): this same bug occurs as well for PNG, GIF, JPEG images, as well as PDF or DejaVu documents, or audio and video files!
• Reverting any file to a previous version does not work (except for their generated PNG thumbnails), and files that must be reverted, because of an unexpected error (e.g. from any user that used the wrong filename and unepectedly overwrote the incorrect file), or because of some users voluntarily damaging file contents with abusive uploads (the revert from the file history also does not work, except for the served thumbnails). This creates severe maintenance problems against user abuses or errors, or for any legitimate fixes that may be desirable and needed.
• Transfering any file from Wikipedia to Commons may in fact transfer an old version that was updated on Wikipedia, where the correct last version will be ''permanently deleted'' just after the transfer (an no more history available in Commons or in Wikipedia to find the correct last version).

This bug is reproducible everywhere (you can try with your own file, upload it, make a correct file description page with the required licences, date, your author name, and relevant categories, then update the file content several times with successive visible edits, look at the file description page and find a logic to the displayed file history, but don't look just at the displayed list of thumbnails: click each version thumbnail to see the full version and you'll see that they don't match...).

Note that I also found another problem that this bug has caused, notably a few battles of file reverts, where two conflicting users had different views of the problem, where they were both right. The problem lies in file histories that is inconsistant and files served are different. Local and global cache consistencies are important. We should avoid these battles. And make sure that wiki admins are aware of the problem, so that they also don't make inconsitant choices caused by inconsistancies of Wikimedia caches (that affect everyone, even if many have not detected or experienced this tricky problem).

Note that the workaround with a ''dummy query string'' such as "?action=purge" at end of the "upload.wikimedia.org" URL may work... sometimes. But more often not at all! Or you may wwant to try using another query string such as "?0", "?1" and so on. But this still does not warranty that the non-decorated normal URL will be correct for everyone (the various Wikimedia's front cache servers are still corrupted, which one is used depends on the location of their ISP, due to the way the CDNs are working). Apparently the front cache servers or backend storage servers are still limiting these dummy query strings to request for the same stored resource. There's no reliable way to purge those corrupted caches in Wikimedia which get out of sync too easily.

Interestingly thumbs work. So when the image you upload is big (like 400x400) the thumb in history is fine (because it is a thumb). Raw images served from upload. domain are a problem.

Possibly Varnish normalizes thumb URLs but doesn't normalize original URLs (which until now did not have any query parameters, other than the Content-Disposition toggle)?

In T425216#12284293, @Tgr wrote:

Possibly Varnish normalizes thumb URLs but doesn't normalize original URLs (which until now did not have any query parameters, other than the Content-Disposition toggle)?

per the previous discussion (T435283#12240228) its the opposite. Mediawiki purged the normalized thumb url, but when purging original asset used a non normalized url to purge.

Today, it took about an hour for the original SVG content of this file to be updated to a new version, for purposes of thumbnail generation (the old SVG was somewhat malformed, which broke thumbnail generation after recent thumbnailing stack upgrades):

https://commons.wikimedia.org/wiki/File:Axis_Communications_logo.svg

All the time, https://commons.wikimedia.org/w/api.php?action=query&titles=File:Axis_Communications_logo.svg&prop=imageinfo&iiprop=sha1|size|timestamp&format=json reported the correct new file size and sha1 checksum, but thumbnails continued to be broken. (They are now correctly rendered, after the mentioned hour-long wait.)

The "original file" link https://upload.wikimedia.org/wikipedia/commons/2/2b/Axis_Communications_logo.svg ("utm…" parameters removed for brevity) still serves the old version in Stockholm, Sweden, where I am, though. If it still not up to date a day from now, I will post an update for information.

Change #1341936 had a related patch set uploaded (by Krinkle; author: Krinkle):

[operations/puppet@production] varnish: Fix thumb.wm.o normalization to match upload.wm.o

https://gerrit.wikimedia.org/r/1341936

Change #1341937 had a related patch set uploaded (by Krinkle; author: Krinkle):

[operations/puppet@production] varnish: Move query string strip for upload.wm.o to pre-purge

https://gerrit.wikimedia.org/r/1341937

Change #1342705 had a related patch set uploaded (by Krinkle; author: Krinkle):

[mediawiki/core@master] WANObjectCache: Migrate internal scheduleAsyncRefresh to touchedCb

https://gerrit.wikimedia.org/r/1342705

In T425216#12278778, @Krinkle wrote:

In T435283#12248179, @BBlack wrote:

[…] IMHO, the right fix for this is to fix the MW side of this to purge the base URL of the original, without the excess params […]

In T435283#12249761, @Bawolff wrote:

I agree that it seems like mediawiki is as at fault here. I think these utm parameters were added relatively recently as part of T414338. cc'ing @Krinkle

I initially leaned towards fixing the Varnish side instead for reasons I'll discuss. But, since both of you suggest otherwise, I looking at it from the MW side first with the intention to fix Varnish afterwards as well for good measure.

[...]

Options 2, 3, or 4 seemed fine with varying compromises however, this last option made me really think about the consequences:

Why should MediaWiki purge a different URL than the canonical URL it uses everywhere in its interfaces?

That would leave MediaWiki purging broken by default on any setup with a cache proxy or CDN in front, unless that CDN is configured the same novel quirk as WMF to strip query params from upload.wikimedia.org on GET but not on PURGE. MediaWiki would promote one link, but then purge a different one.

We apply almost all of our normalizations pre-purge in Varnish VCL. E.g. hostname normalization, URL path normalization, URL query parameter sorting. Why should this be any different?

I was under the impression (implicitly, due to how our VCL looks today) that the utm parameter is *not* part of the canonical URL. If it is, then we shouldn't be stripping it for requests, and the base URL without the query string should be a 404, right? Clearly we're struggling to define what canonical means here between the two sides.

The way I see things (which is not to say I'm right, but I think we need to understand where and why I'm wrong to get at the bottom of this), the goals at the CDN layer here are:

1. To not pointlessly cache N copies of the same image due to some analytics/defenses/etc -related parameter that's not canonical to the content URL (which is why we're stripping the param in the request flow now, I assume)
2. To purge the one copy of content reliably when asked (if the base URL is canonical and the base URL is purged, and we strip requests down to the base URL, this should all Just Work)
3. To not copy any more application-specific business logic into CDN edge logic than we absolutely have to (because it binds them together and makes future changes more complicated).

If for whatever reason, the answer ends up actually being "have the CDN strip query params from PURGE URLs", then we at least need to patch that into both cache layers, not just Varnish.

I run into this problem when trying to reupload this file. The workaround with uploading a nulledit (just changing metadata) didn't help. While I get the outdated version in Germany, I am able to download the correct version from a server in New York City. Try to download the original file. The correct sha256sum of the most recent upload is 8696b9c48d60c0b39a1324cd358c3eda77f93166c7ed374232c3c4baa816bae4. The very first upload has defb9c72963847fbba65fb8ec955dd9fc3d3a0394d5724753c9c22af70ae9ddf.

In T425216#12351266, @BBlack wrote:

To not copy any more application-specific business logic into CDN edge logic than we absolutely have to (because it binds them together and makes future changes more complicated).

Doesn't that argument cut the other way? If the edge logic applies different normalization to GETs and PURGEs, the application needs to know about edge normalization logic and pre-apply it to purges. If the rules are identical, the application doesn't need to care about it and can just send the same URL that it would use for fetching the thumbnail.

In T425216#12351266, @BBlack wrote:

I was under the impression (implicitly, due to how our VCL looks today) that the utm parameter is *not* part of the canonical URL.

The VCL we have predates provenance. When I reviewed it before enabling provenance, I incorrectly concluded that Varnish always normalizes a request for the same URL in the same way in vcl_recv. I'm familar with cluster_fe_recv_pre_purge split from when I worked on the mobile domain, but I failed to notice that media_clean_urls is called from cluster_fe_recv instead of cluster_fe_recv_pre_purge.

It is actually very close to being true today: host normalization, article path normalization, upload path normalization, query order sorting normalization are applied consistently. It seems that upload query stripping is the only outlier today.

Is this an intentional distinction that provides an important benefit or value?

[…] the base URL without the query string should be a 404, right?

If I understand correctly, you're asking whether two URLs with the same path but more or fewer parameters could both be valid and distinct canonical URLs. My answer would be: Yes. It seems reasonable for /foo, /foo?bar=x, and /foo?bar=x&quux=y to be distinct canonical URLs. For example, if they are optional parameters set to a non-default value. The canonical URL for /foo?bar=default&quux=default would be /foo.

Having said that, I'm not aware of any such cases in MediaWiki today. Why do you ask?

To not copy any more application-specific business logic into CDN edge logic than we absolutely have to (because it binds them together and makes future changes more complicated).

If you mean that stripping query parameters in general is business logic, I agree. It is an optimisation we made in the CDN over a decade ago to improve caching, and cache performance optimisations are business logic at our scale. (The alternative would be for MediaWiki to send 10,000 purges per thumbnail width, 1000 wikis as provenance sources x 10 callers as provenance campaigns, but that isn't viable for us. Even as a technically-correct "default" I don't think there'd be an audience for it. It seems reasonable to document that if you want to enable CDN caching with purge support for a MediaWiki site, you have to strip query parameters from image URLs.)

If you mean that what we do today (stripping on GET but not on PURGE) is not copying business logic, but that stripping it always is copying business logic, I disagree. I imagine this makes things harder to reason about for VCL maintainers as well? Is there a compelling reason for MediaWiki to support such an asymetric configuration?.

Clearly we're struggling to define what canonical means here between the two sides.

Indeed, this is true.

MediaWiki has a single method for the "original" file URL (File::getUrl). This method takes no parameters, and the class has no config options to change its return value. The returned value always includes provenance parameters, and they're always the same. One could argue this is the canonical URL for the original. And this is how it is treated today, both in terms of what MediaWiki links to in its UI/API, and in terms of what it purges.

MediaWiki has a deep and variable method for thumbnail URLs (File::transform), which takes parameters that greatly vary the provenance parameters it adds. Changing the purge for thumbnails to include parameters as well may appear "consistent", but it wouldn't be useful in practice, because we'd be forced to pick one arbitrary variant, and none are particularly more canonical than any other. It's hard to argue that the param-less URL is canonical, because by design it is not a URL we advertise anywhere or want to encourage use of. But, it's a neutral good I suppose, given there are only bad choices, and so it becomes a requirement that query params are stripped in order to support purging with CDN caching on thumbnails.

Then here comes the twist: one of the thumbnail outputs is to re-use the original file (if the original is in a websafe format like JPEG, and if the original is so small e.g. 300px, that a 500px thumbnail would scale it up, then we use the original as the thumbnail), in which case it gets the same highly variable params as thumbnail URLs appended. This twist doesn't apply to most files. For most files there's only one widely used original URL, and it's either not websafe or large enough that article usage will generate a thumbnail file. As such, while there is a canonical URL for the original, for a subset of files, there exist non-canonical original URLs as well.

This give us two choices:

• Status quo: MediaWiki sends purges for canonical URLs where there is one. For the case where there is no canonical URL (thumbnails) it purges a generic base URL without query parameters. This is what we do. This means caching and purging mostly works out of the box with no specific normalization required, if you only care about first-party minted URLs, except for purging thumbnails where query param stripping is required for purging to work. Today, because the asymetric stripping, the purges for the originals are ineffective. If there wasn't any stripping, then at least purging originals would work for the canonical case. On the flipside, thanks to stripping being there at least on non-purge requests, it does mean thumbnails enjoy purging even for URLs made by third-party or cross-wikis various other query strings appended.
• Change purging of original: We could add a bypass somewhere to expose the un-parameterized version of the original File url, and purge this. This reduces symmetry in what MW links and what MW purges and increases the surface area where CDN normalization is a functional requirement imposed by MediaWiki.

I was originally planning to do both, but there is not a clear benefit. It doesn't change the CDN requirements, unless we include pre/post purge distinction in MediaWiki's CDN requirements.

Change #1344703 had a related patch set uploaded (by Krinkle; author: Krinkle):

[mediawiki/core@master] File: Implement LocalFile::getUrlForPurge workaround for caching

https://gerrit.wikimedia.org/r/1344703

Still not solved, and signaled with more details on https://commons.wikimedia.org/wiki/Commons:Village_pump/Archive/2026/09
This bug is persistant, the cache can even keep a version that was completely deleted, before being replaced by an entirely new original file.
It does not affect at all thumbnails (that are generated on deman with a few delay after the effective upload was stored).
But jut after the upload, the original is not visible on the backstore, and the frontcache attempts to load it: it fails, and instantly renews the old version it has in cache.
Trying to upload a file with an internal "nulledit" (e.g. adding or removing a non-significant whitespace in a SVG, or changing some internal metadata with a minor edit) has NO effect at all.
Definitely the upload wizard is incorrectly synchronized when it clears the cache before the file is effectively stored on the backend. This does not affect the thumbnail generator which DOES not use the frontend cache but requests the original file from the backend (but will wait correctly as long as it's still not ready). I don't know how to get the effective original through the cache, which never reflects what is in the storage backend. It is then just impossible to update ANY file (in any media format), except the thumbnails, the frontcache ignores any attmpt to purge it to force it to reload the correct file from the storage backend.

Change #1344703 merged by jenkins-bot:

[mediawiki/core@master] File: Implement LocalFile::getUrlForPurge workaround for caching

https://gerrit.wikimedia.org/r/1344703

Change #1345216 had a related patch set uploaded (by Krinkle; author: Krinkle):

[mediawiki/core@wmf/1.47.0-wmf.21] File: Implement LocalFile::getUrlForPurge workaround for caching

https://gerrit.wikimedia.org/r/1345216

In T425216#12360080, @Krinkle wrote:

[,,, re: canonical ...]
Is this an intentional distinction that provides an important benefit or value?

[…] the base URL without the query string should be a 404, right?

If I understand correctly, you're asking whether two URLs with the same path but more or fewer parameters could both be valid and distinct canonical URLs.

What I was trying to get to here, perhaps unsuccessfully, was the idea that "canonical" for us means: the one true address of this content (well, conceptually, subject to replacement), so that we can store exactly one copy in a cache (and purge it by that same name). The fact that it's supposed to be fetchable by other names (e.g. provenance-less) is a wrinkle in this view.

MediaWiki has a single method for the "original" file URL (File::getUrl). This method takes no parameters, and the class has no config options to change its return value. The returned value always includes provenance parameters, and they're always the same. One could argue this is the canonical URL for the original. And this is how it is treated today, both in terms of what MediaWiki links to in its UI/API, and in terms of what it purges.

Does the provenance ever change over time?

Either way, it sounds like patching over this in the CDN is easier. But if we have to do that, then we need to do it both layers or it will still be ineffective (the request URLs flow down from Varnish->ATS already stripped of provenance, but PURGE arrives at ATS independently).

Change #1345282 had a related patch set uploaded (by Krinkle; author: Krinkle):

[operations/puppet@production] trafficserver: Document and test normalize-path

https://gerrit.wikimedia.org/r/1345282

Change #1345216 merged by jenkins-bot:

[mediawiki/core@wmf/1.47.0-wmf.21] File: Implement LocalFile::getUrlForPurge workaround for caching

https://gerrit.wikimedia.org/r/1345216

Mentioned in SAL (#wikimedia-operations) [2026-09-29T13:56:40Z] <krinkle@deploy1003> Started scap sync-world: Backport for [[gerrit:1345216|File: Implement LocalFile::getUrlForPurge workaround for caching (T425216)]], [[gerrit:rAPIW1344298e2265|ProductionServices: Simplify indirect parsercache-dbs stub (T362786 T437209)]]

Mentioned in SAL (#wikimedia-operations) [2026-09-29T14:00:51Z] <krinkle@deploy1003> krinkle: Backport for [[gerrit:1345216|File: Implement LocalFile::getUrlForPurge workaround for caching (T425216)]], [[gerrit:rAPIW1344298e2265|ProductionServices: Simplify indirect parsercache-dbs stub (T362786 T437209)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

LGTM now, for some definition of "good". 169 URLs when purging https://commons.wikimedia.org/wiki/File:Example.svg. No provenance params though.

channel: squid
message:
MediaWiki\Deferred\CdnCacheUpdate::purge:
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/120px-Example.svg.jpg
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/120px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/120px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/1280px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/1280px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/1920px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/1920px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/200px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/20px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/20px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/250px-Example.svg.gif
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/250px-Example.svg.jpg
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/250px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/330px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/330px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/3840px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/3840px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/40px-Example.svg.gif
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/40px-Example.svg.jpg
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/40px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/40px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/500px-Example.svg.gif
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/500px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/500px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/60px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/60px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/960px-Example.svg.gif
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/960px-Example.svg.jpg
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/960px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/960px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langain-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langain-250px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langain-330px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langar-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langar-250px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langaz-60px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langaz-60px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langbe-tarask-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langbe-tarask-250px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langcs-120px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langcs-120px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langcs-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langcs-250px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langcs-500px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langde-120px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langde-120px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langde-20px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langde-20px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langde-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langde-250px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langde-330px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langde-330px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langde-500px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langde-500px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langen-gb-120px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langen-gb-120px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langen-gb-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langen-gb-250px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langen-gb-330px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langen-gb-500px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langen-gb-960px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langes-120px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langes-120px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langes-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langes-250px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langes-330px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langes-960px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langfa-120px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langfa-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langfa-250px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langfa-330px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langfa-330px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langfr-120px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langfr-120px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langfr-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langfr-250px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langfr-330px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langfr-330px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langfr-500px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langfr-500px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langfr-960px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langfr-960px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langhr-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langhu-120px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langhu-120px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langit-120px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langit-120px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langit-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langit-250px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langja-120px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langja-120px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langja-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langja-250px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langja-330px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langja-500px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langja-500px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langja-960px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langko-120px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langko-120px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langko-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langko-330px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langko-500px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langko-960px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langmzn-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langmzn-250px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langnl-120px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langnl-120px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langpl-120px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langpl-120px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langpl-1280px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langpl-1280px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langru-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langru-250px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langru-500px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langru-60px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langsq-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langsq-250px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langsq-500px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langsq-500px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langtr-120px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langtr-120px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/languz-120px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/languz-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/languz-330px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/languz-40px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/languz-500px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/languz-60px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/languz-960px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langvlc-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langyue-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langyue-250px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langzh-500px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langzh-500px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langzh-cn-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langzh-cn-250px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langzh-tw-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langzh-tw-250px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/500px-Example.svg.jpg
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langen-ca-120px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langen-ca-120px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langja-40px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langja-40px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langja-960px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langml-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langml-250px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langms-120px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langms-120px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langpt-br-120px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langpt-br-120px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langth-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langth-250px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/languk-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/languk-250px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langzh-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langzh-250px-Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langzh-hans-250px-Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/langzh-hans-250px-Example.svg.webp
https://upload.wikimedia.org/wikipedia/commons/8/84/Example.svg
https://thumb.wikimedia.org/wikipedia/commons/thumb/archive/8/84/20080323205329%21Example.svg/120px-20080323205329%21Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/archive/8/84/20080323205329%21Example.svg/120px-20080323205329%21Example.svg.webp
https://thumb.wikimedia.org/wikipedia/commons/thumb/archive/8/84/20080323205329%21Example.svg/250px-20080323205329%21Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/archive/8/84/20080323205329%21Example.svg/250px-20080323205329%21Example.svg.webp
https://upload.wikimedia.org/wikipedia/commons/archive/8/84/20080323205329%21Example.svg
https://thumb.wikimedia.org/wikipedia/commons/thumb/archive/8/84/20070525200009%21Example.svg/120px-20070525200009%21Example.svg.png
https://thumb.wikimedia.org/wikipedia/commons/thumb/archive/8/84/20070525200009%21Example.svg/120px-20070525200009%21Example.svg.webp
https://upload.wikimedia.org/wikipedia/commons/archive/8/84/20070525200009%21Example.svg
https://commons.wikimedia.org/wiki/File:Example.svg
https://commons.wikimedia.org/w/index.php?title=File:Example.svg&action=history

Most importantly number 159:

https://upload.wikimedia.org/wikipedia/commons/8/84/Example.svg

Mentioned in SAL (#wikimedia-operations) [2026-09-29T14:11:06Z] <krinkle@deploy1003> Finished scap sync-world: Backport for [[gerrit:1345216|File: Implement LocalFile::getUrlForPurge workaround for caching (T425216)]], [[gerrit:rAPIW1344298e2265|ProductionServices: Simplify indirect parsercache-dbs stub (T362786 T437209)]] (duration: 14m 26s)