Page MenuHomePhabricator

De-fragment thumbnail sizes in mediawiki
Closed, ResolvedPublic

Description

Currently, users can set any thumbnail sizes in mediawiki and we will serve them the exact same size. This has caused many issues: T211661: Automatically clean up unused thumbnails in Swift T345334: Cache thumbs in our caching infrastructure (e.g. ATS) and many more tickets connected to them.

A rather simple improvement to this is to expand pregen sizes (in mw) and when users set the thumbsize when editing (e.g. if they pick 110px), we pick the smallest thumbsize that's larger than the picked size (in this case: 120px). Set the image URL to that size but resize the <img> tag to the requested size (110px in this example).

For what pregen sizes we can pick, here are some data of most used sizes:

cache statussize (px)hits (in a day)
hit-front220155602049
hit-front440135746095
hit-front40111905238
hit-front16097151635
hit-front2095060013
hit-front2393020718
hit-front6082391195
hit-front96077506680
hit-front3070646190
hit-front12068306081
hit-front4553407704
hit-front33049912716
hit-front2249074909
hit-front30048777979
hit-front4448309778
hit-local22048146954
hit-front4647160522
hit-front5044400026
hit-front3543559169
hit-local44043416242
hit-front20041122192
hit-front18839760280
hit-front48033331526
hit-local16033202649
hit-local12032572116
hit-front1531546759
hit-front10030324205
hit-local33028760556
hit-front50027901900
hit-front15026586327

We don't need to change anything in swift/thumbor/ATS/varnish/etc. but it will reduce a lot of cache fragmentation and load on thumbor and storage on swift. It'll improve user experience by serving more from edge caches than backends.

This probably should be rolled out gradually and wiki by wiki (by a config flag)

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
mediawiki/coremaster+40 -5
operations/mediawiki-configmaster+1 -6
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/deployment-chartsmaster+4 -4
operations/deployment-chartsmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
mediawiki/corewmf/1.44.0-wmf.20+10 -0
mediawiki/coremaster+10 -0
mediawiki/coremaster+12 -1
operations/mediawiki-configmaster+2 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+5 -2
operations/mediawiki-configmaster+7 -0
operations/mediawiki-configmaster+8 -0
mediawiki/coremaster+183 -1
Show related patches Customize query in gerrit

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change #1133354 merged by jenkins-bot:

[operations/mediawiki-config@master] Bump thumbnail steps to 60%

https://gerrit.wikimedia.org/r/1133354

Mentioned in SAL (#wikimedia-operations) [2025-04-02T10:53:40Z] <ladsgroup@deploy1003> Started scap sync-world: Backport for [[gerrit:1133354|Bump thumbnail steps to 60% (T360589)]]

Mentioned in SAL (#wikimedia-operations) [2025-04-02T11:00:14Z] <ladsgroup@deploy1003> ladsgroup: Backport for [[gerrit:1133354|Bump thumbnail steps to 60% (T360589)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2025-04-02T11:08:52Z] <ladsgroup@deploy1003> Finished scap sync-world: Backport for [[gerrit:1133354|Bump thumbnail steps to 60% (T360589)]] (duration: 15m 11s)

Change #1133862 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[operations/mediawiki-config@master] Bump thumbnail steps to 65%

https://gerrit.wikimedia.org/r/1133862

Change #1133862 merged by jenkins-bot:

[operations/mediawiki-config@master] Bump thumbnail steps to 65%

https://gerrit.wikimedia.org/r/1133862

Mentioned in SAL (#wikimedia-operations) [2025-04-03T10:46:10Z] <ladsgroup@deploy1003> Started scap sync-world: Backport for [[gerrit:1133862|Bump thumbnail steps to 65% (T360589)]]

Mentioned in SAL (#wikimedia-operations) [2025-04-03T10:53:18Z] <ladsgroup@deploy1003> ladsgroup: Backport for [[gerrit:1133862|Bump thumbnail steps to 65% (T360589)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2025-04-03T11:02:45Z] <ladsgroup@deploy1003> Finished scap sync-world: Backport for [[gerrit:1133862|Bump thumbnail steps to 65% (T360589)]] (duration: 16m 34s)

I did a quick check today:

spark-sql (default)> select cache_status, split(split(uri_path, '/')[7], 'px-')[0] as thumbsize, count(*) as hitcount from wmf.webrequest where webrequest_source = 'upload' and year = 2025 and month = 4 and day = 2 and hour = 23 and http_status = 200 and uri_path like '/wikipedia/%/thumb/%' group by split(split(uri_path, '/')[7], 'px-')[0], cache_status order by hitcount desc limit 500;
cache_status	thumbsize	hitcount
hit-front	250	7720918
hit-front	40	7667577
hit-front	60	7124956
hit-front	120	5022703
hit-front	500	4860438
hit-front	330	3439313
hit-front	20	3110168
hit-front	440	2575422
hit-front	220	2218726
hit-local	250	1890534
hit-front	23	1479510
hit-front	160	1336626
hit-local	500	1302593
hit-local	330	1291469
hit-local	120	1217313
hit-front	46	906487
hit-front	45	897540
...

The standard sizes are clearly going up. What I find quite interesting though is that the ratio of hit-front to hit-local for thumbnail sizes is getting quite high. For example, in the task description. The ratio of hit-front to hit-local for 220px was 3.23 but the same ratio for 250px today is 4.08. You can see the jump across all sizes. 120px went from 2.10 to 4.13. 330px has gone from 1.74 to 2.66 and so on. This means there are much better cache efficiency and reduced latency in showing images to the users.

Change #1134651 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[operations/mediawiki-config@master] Bump thumbnail steps to 70%

https://gerrit.wikimedia.org/r/1134651

Change #1134651 merged by jenkins-bot:

[operations/mediawiki-config@master] Bump thumbnail steps to 70%

https://gerrit.wikimedia.org/r/1134651

Mentioned in SAL (#wikimedia-operations) [2025-04-07T10:32:14Z] <ladsgroup@deploy1003> Started scap sync-world: Backport for [[gerrit:1134651|Bump thumbnail steps to 70% (T360589)]]

Mentioned in SAL (#wikimedia-operations) [2025-04-07T10:37:14Z] <ladsgroup@deploy1003> ladsgroup: Backport for [[gerrit:1134651|Bump thumbnail steps to 70% (T360589)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2025-04-07T10:46:37Z] <ladsgroup@deploy1003> Finished scap sync-world: Backport for [[gerrit:1134651|Bump thumbnail steps to 70% (T360589)]] (duration: 14m 22s)

Change #1134999 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[operations/mediawiki-config@master] Bump thumbnail steps to 75%

https://gerrit.wikimedia.org/r/1134999

Change #1134999 merged by jenkins-bot:

[operations/mediawiki-config@master] Bump thumbnail steps to 75%

https://gerrit.wikimedia.org/r/1134999

Mentioned in SAL (#wikimedia-operations) [2025-04-08T10:57:22Z] <ladsgroup@deploy1003> Started scap sync-world: Backport for [[gerrit:1134999|Bump thumbnail steps to 75% (T360589)]]

Mentioned in SAL (#wikimedia-operations) [2025-04-08T11:04:36Z] <ladsgroup@deploy1003> ladsgroup: Backport for [[gerrit:1134999|Bump thumbnail steps to 75% (T360589)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2025-04-08T11:13:57Z] <ladsgroup@deploy1003> Finished scap sync-world: Backport for [[gerrit:1134999|Bump thumbnail steps to 75% (T360589)]] (duration: 16m 35s)

Change #1135392 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[operations/mediawiki-config@master] Bump thumbnail steps to 80%

https://gerrit.wikimedia.org/r/1135392

Change #1135392 merged by jenkins-bot:

[operations/mediawiki-config@master] Bump thumbnail steps to 80%

https://gerrit.wikimedia.org/r/1135392

Mentioned in SAL (#wikimedia-operations) [2025-04-09T10:04:19Z] <ladsgroup@deploy1003> Started scap sync-world: Backport for [[gerrit:1135392|Bump thumbnail steps to 80% (T360589)]]

Mentioned in SAL (#wikimedia-operations) [2025-04-09T10:12:00Z] <ladsgroup@deploy1003> ladsgroup: Backport for [[gerrit:1135392|Bump thumbnail steps to 80% (T360589)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2025-04-09T10:18:39Z] <ladsgroup@deploy1003> Finished scap sync-world: Backport for [[gerrit:1135392|Bump thumbnail steps to 80% (T360589)]] (duration: 14m 19s)

Change #1135688 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[operations/mediawiki-config@master] Bump thumbnail steps to 85%

https://gerrit.wikimedia.org/r/1135688

Change #1135688 merged by jenkins-bot:

[operations/mediawiki-config@master] Bump thumbnail steps to 85%

https://gerrit.wikimedia.org/r/1135688

Mentioned in SAL (#wikimedia-operations) [2025-04-10T11:10:09Z] <ladsgroup@deploy1003> Started scap sync-world: Backport for [[gerrit:1135688|Bump thumbnail steps to 85% (T360589)]]

Mentioned in SAL (#wikimedia-operations) [2025-04-10T11:15:58Z] <ladsgroup@deploy1003> ladsgroup: Backport for [[gerrit:1135688|Bump thumbnail steps to 85% (T360589)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2025-04-10T11:26:29Z] <ladsgroup@deploy1003> Finished scap sync-world: Backport for [[gerrit:1135688|Bump thumbnail steps to 85% (T360589)]] (duration: 16m 20s)

JJMC89 subscribed.

@Ladsgroup: It looks like the work here lead to a breaking change in the impageinfo API. The iiurlwidth and iiurlheight parameters no longer function as documented, and the returned thumbwidth and thumbheight do not match the thumburl.

Change #1136339 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[operations/mediawiki-config@master] Bump thumbnail steps to 90%

https://gerrit.wikimedia.org/r/1136339

Change #1136339 merged by jenkins-bot:

[operations/mediawiki-config@master] Bump thumbnail steps to 90%

https://gerrit.wikimedia.org/r/1136339

@Ladsgroup: It looks like the work here lead to a breaking change in the impageinfo API. The iiurlwidth and iiurlheight parameters no longer function as documented, and the returned thumbwidth and thumbheight do not match the thumburl.

Thanks. I'll comment there.

Mentioned in SAL (#wikimedia-operations) [2025-04-14T10:41:51Z] <ladsgroup@deploy1003> Started scap sync-world: Backport for [[gerrit:1136339|Bump thumbnail steps to 90% (T360589)]], [[gerrit:1135835|CommonSettings: remove outdated SecurePoll comment (T209892)]]

Mentioned in SAL (#wikimedia-operations) [2025-04-14T10:47:22Z] <ladsgroup@deploy1003> ladsgroup, novemlinguae: Backport for [[gerrit:1136339|Bump thumbnail steps to 90% (T360589)]], [[gerrit:1135835|CommonSettings: remove outdated SecurePoll comment (T209892)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2025-04-14T10:59:18Z] <ladsgroup@deploy1003> Finished scap sync-world: Backport for [[gerrit:1136339|Bump thumbnail steps to 90% (T360589)]], [[gerrit:1135835|CommonSettings: remove outdated SecurePoll comment (T209892)]] (duration: 17m 26s)

Change #1136670 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[operations/mediawiki-config@master] Bump thumbnail steps to 95%

https://gerrit.wikimedia.org/r/1136670

Change #1136670 merged by jenkins-bot:

[operations/mediawiki-config@master] Bump thumbnail steps to 95%

https://gerrit.wikimedia.org/r/1136670

Mentioned in SAL (#wikimedia-operations) [2025-04-15T10:19:07Z] <ladsgroup@deploy1003> Started scap sync-world: Backport for [[gerrit:1136670|Bump thumbnail steps to 95% (T360589)]]

Mentioned in SAL (#wikimedia-operations) [2025-04-15T10:33:18Z] <ladsgroup@deploy1003> sync-world aborted: Backport for [[gerrit:1136670|Bump thumbnail steps to 95% (T360589)]] (duration: 14m 11s)

Mentioned in SAL (#wikimedia-operations) [2025-04-15T10:34:35Z] <ladsgroup@deploy1003> Started scap sync-world: Backport for [[gerrit:1136670|Bump thumbnail steps to 95% (T360589)]]

Mentioned in SAL (#wikimedia-operations) [2025-04-15T10:39:44Z] <ladsgroup@deploy1003> sync-world aborted: Backport for [[gerrit:1136670|Bump thumbnail steps to 95% (T360589)]] (duration: 05m 08s)

Change #1136963 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[operations/mediawiki-config@master] Bump thumbnail steps to 100%

https://gerrit.wikimedia.org/r/1136963

Change #1136963 merged by jenkins-bot:

[operations/mediawiki-config@master] Bump thumbnail steps to 100%

https://gerrit.wikimedia.org/r/1136963

Mentioned in SAL (#wikimedia-operations) [2025-04-16T08:46:12Z] <ladsgroup@deploy1003> Started scap sync-world: Backport for [[gerrit:1136963|Bump thumbnail steps to 100% (T360589)]]

Mentioned in SAL (#wikimedia-operations) [2025-04-16T08:58:02Z] <ladsgroup@deploy1003> ladsgroup: Backport for [[gerrit:1136963|Bump thumbnail steps to 100% (T360589)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2025-04-16T09:15:02Z] <ladsgroup@deploy1003> ladsgroup: Backport for [[gerrit:1136963|Bump thumbnail steps to 100% (T360589)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2025-04-16T09:22:04Z] <ladsgroup@deploy1003> Finished scap sync-world: Backport for [[gerrit:1136963|Bump thumbnail steps to 100% (T360589)]] (duration: 19m 05s)

Ladsgroup claimed this task.

It will take a while to be fully propagated since we have CDN cache and ParserCache but the bulk of it is done now. After this we will do a lot more work in improving the infra.

Okay, Now that most caches have been invalidated. I tried to measure the impact of this. I collected hour by hour cache status of hits to text and upload cluster starting from Nov 2024 until two days ago. Then I used hits to text cluster as basis to build three different linear regressions (one using hit-front, hit-local, miss of text cluster to predict hit-front of upload, another using the same variables to predict hit-local of upload, etc.). This is called synthetic control. I used the data from Nov to 1st of March to produce the regressions. I ignored anything bigger than 3*sigma to ignore spikes (this is a usual tool to get rid of unnatural outliers). The result are quite interesting.

Here is the data vs regression for upload hit-local:

i_1_1.png (1×1 px, 166 KB)

Here is the data vs regression for upload miss (aka calls leading to swift or thumbor):

i_1_2.png (969×1 px, 160 KB)

(x is day, y is hits in that day)

Clearly it has moved a lot to hit-front but funnily enough, hit-front doesn't show the impact:

i_1_0.png (969×1 px, 163 KB)

I was confused but than I realized hit-front is one order of magnitude larger so any move from hit-local or miss will just get lost in the noise. I try to build a regression model with percentage of hit-front, maybe that'd show itself more clearly but this is still a clear result of improvements caused by the thumb steps work (code: P76278, data: P76279)

If you want numbers, if we apply the synthetic control on May hit rates, every day ~614,000±123,000 less hit-local are happening because of thumbnail steps (24±7% reduction). In terms of misses, 221,000±105,000 requests every day are being removed from hitting swift/thumbor which is 23±10% reduction.

FWIW, I added a notification about this breaking change on https://www.mediawiki.org/wiki/API:Imageinfo - rephrase away if you think it is unclear.

(Background. This change broke functionality in one of my MediaWiki extensions, which can fetch data from Wikimedia Commons' API service and return IIIF-compliant data so that for instance, images can be accessed in a deep-zoom viewer like OpenSeadragon. IIIF requires that the size returned is the exact size requested (https://iiif.io/api/image/3.0/#53-sizes) and for that to happen, I used to rely on the imageinfo API. Now that it cannot be relied on, I fixed things by keeping the same API request but reconstructing the URL to get the desired size of the image. I'm glad that direct access to thumbnail URLs still works!)

The Dutch community has discussed what thumb size should be, and the thumb size is with consensus established on 260px. Changing it to 250px is against this consensus and 250px considered too small for the Dutch Wikipedia.

@Romaine the change was done under T408715 not this task, FWIW. If you want to pursue it, it'd be best done there.

AFAICT the complaint is that the 250 size is wrong for infoboxes? If I look at Dalai Lama (NL) for example, the infobox there has the 330px thumb browser-scaled to 266px wide, whereas Dalai Lama (EN) has 250px in the infobox.

Is the NL argument that everyone else is wrong about the appropriate size for the image in the infobox? Or that the NL infoboxes are a different style to the standard one and thus warrant a different thumb size? Or something else?