Page MenuHomePhabricator

Propose a new set of standard thumbnail sizes
Closed, ResolvedPublic

Description

We've collected the set of thumbnail sizes that are currently requested by our infrastructure and clients (T408715), and looked at how frequently these sizes (and others) occur in thumbnail requests currently seen at the edge of our CDN (T410304). Now it's time to propose a rationalised set of thumbnail sizes with a view to standardising on using only these sizes:

20, 40, 60, 120, 250, 330, 500, 960, 1280, 1920, 3840
Of which, pre-generate 250, 330, 500,960 none

Rationale:

Pre-generate 4 sizes as before, but pick four popular and larger thumb sizes to pre-generate.
Just pre-generate the standard commons file page size (which is what users will see once they've uploaded something) by way of making sure the uploaded image is thumbnailable.
As we're moving towards actually caching (rather than storing nearly-indefinitely) thumbnails (cf T345334), don't bother pre-generating any sizes.

Given modern display sizes, we want some steps between 960 and full size.

20, 40, 60, 120, 250, 330, 500, 960 - the existing $wgThumbnailSteps
1280 - full-screen Gallery view on Android (and header & gallery images on retina devices on iOS)
1920 - HD (download option in MultimediaViewer)
3840 - 4K (download option in MultimediaViewer)

Questions:
iOS currently uses 2x320 = 640 on retina devices, which would end up being 960-downscaled. Is that too wasteful? cf T412161
Does iOS actually use WMFImageWidthExtraLarge (1280/2560) and/or WMFImageWidthExtraExtraLarge (1920/3840)?
Do we want to also allow 2560? Or will downscaling-from-3840 be OK?

It's worth noting that a fair amount of work is needed (including on some extensions currently in maintenance-only mode) to standardise our usage onto these sizes; this will need prioritisation and communication to the relevant communities.

Event Timeline

iOS currently uses 2x320 = 640 on retina devices, which would end up being 960-downscaled. Is that too wasteful? cf T412161

I don't think many current phones still stick to 320 css pixels wide (640 device pixels at 2x density) anymore, so I think we'd end up picking the 960 anyway if fitting actual device size without scaling up. I'd be kind of inclined to keep a 640 but not sure if we'd actually use it on real phones... (May need to double-check the scaling logic.)

Does iOS actually use WMFImageWidthExtraLarge (1280/2560) and/or WMFImageWidthExtraExtraLarge (1920/3840)?

If the mobile view is used on iPadOS I believe you'll find those larger widths used in the mobile version of the media viewer.

Do we want to also allow 2560? Or will downscaling-from-3840 be OK?

On a not-quite-full-screen window on a 4k desktop, 2560 or 2880 looks pretty good and will be about half the download size of 3840, so I'd consider keeping a 2560 step for desktop media viewer.

This ticket is touching on something that has been bothering me for some time. The thumbnail steps are "20, 40, 60, 120, 250, 330, 500, 960" and we are expanding that to be the "standard sizes" and that's a misnomer. I'd say what "standard" here means even? My suggestion is this: We build a set of "allowed" sizes that'd the universal set. Everything else will be the subset of it for different usecases. For example, there is possibly no reason whatsoever for us to allow thumbnailing to 3840px width in articles (they have max width of ~960px and even with retina, it's double the size) but it makes sense to allow download of it for wallpapers and so on.

I have a hot take regarding pregen sizes: We should completely remove them. The number of requests to those sizes is quite low comparatively, our infra can make thumbnail out of intermediate sizes (maybe it's not enabled yet?) so I haven't seen it being slow and even if it's slow, it should be created on the fly from intermediate sizes not wasting this many cpu cycles and storage for every file uploaded in case 1% of them might be hit by someone wanting to keep them as wallpaper (as clearly shown by the numbers). 250px doesn't need to be pregen because the moment it's used in any page, it gets created immediately so no need to pregen it.

i.e. My suggestion: Allow 1280px, 1920px, 3840px. but don't pregen it. Don't call it "standard". Just add it to the allow list. It could have a different ratelimit bucket than "standard" thumb sizes (thumbnail steps) but these images are costly and should be e.g. per-IP rate limited regardless. The same way edits are rate-limited. We allow it, just not too much.

Does iOS actually use WMFImageWidthExtraLarge (1280/2560) and/or WMFImageWidthExtraExtraLarge (1920/3840)?

As broke said, for phones these values don't make any sense. The width of a usual phone is ~300px, with retina you have 600px. Maybe that could be a thing in iPads but I don't see it being really different.

It's worth noting that a fair amount of work is needed (including on some extensions currently in maintenance-only mode) to standardise our usage onto these sizes; this will need prioritisation and communication to the relevant communities.

Which extensions or places? Most of them go through thumbnail steps regardless.

Regarding the iOS questions, I think larger iPads are something of a concern; T412161 is going to do some testing in that area.

Extensions-wise, there's at least popups (which has been coded now to pick standard sizes, but as we saw bypasses the MW round-up-to-thumbnailsteps logic), and MultimediaViewer (which could use some work to offer sizes that match the standard set for download) which will be impacted. I'd be surprised if there aren't others, given the sheer number of sizes being requested.

While I see where you're going, I don't think here is the place to get too hung up on terminology; we can call them canonical or standard or allowed (or something else), make sure we're broadly happy with them and then think about how we're going to move towards only supporting these sizes.

I don't feel strongly about pregeneration - the sizes I suggest account for 42% of requests between them, so they seemed a reasonable set if we were going to continue to pregenerate some sizes (and, I think you'll agree, a marked improvement on the current set which account for only 2% between them). I wonder if it's at least worth generating the default-on-a-commons-file-page (which looks to be 960-scaled-to-800 if I check the POTD page when not logged in) to pick up uploads of images that can't be thumbnailed at all at upload time? It might even make sense for commons to default to a size that actually matched a thumbnail step.

Whatever we call it, it would be nice to streamline a couple of larger sizes as well.
I just noticed that 1.5x & 2x srcset for the big file page version, are 1200 & 1600, and MultimediaViewer jumps to 1280 & 2560.
I would expect the file page sizes to have a comparatively high likelihood of already having been rendered (immediately post-upload to check on their work), yet it's not really being used.


In terms of pre-generation, I don't think we should look at the request %, as only the first request matters (from that point on, it has been generated) - the overall file coverage (how many files were requested in this size) matters more.

I agree that the current list of pre-generated thumbs doesn't make a lot of sense, but I would argue that smaller sizes would actually be more worth pre-generating:

  • these appear in longs lists of thumbnails (e.g. search) - there are a lot of them all at once, so odds of users getting rate limited through normal use are higher
  • they're more likely to have broader usage: I would expect the "best looking" images in the "most popular" articles to yield the overwhelming majority of large-width requests, whereas all of the less-popular images are almost just as likely to also show up a lot (in e.g. search results)

FWIW:

  • Search autocomplete thumbnails are displayed at 40x40 (object-fit: cover, so potentially wider) - 60 seems the commonly requested width
  • Special:Search thumbnails are 90x90 (object-fit: cover, so potentially wider) - 60 (3:2), 120 (4:3, 1:1 & 3:4) and 250 (2:3) seem most common here
  • Special:MediaSearch scales based on height (180px), so could also have any (standardized) width - 120 (3:2), 250 (4:3, 1:1 & 3:4) and 330 (2:3) seem most common here
  • (I probably missed a couple of cases)

If I had to make a bet on what size any random image would most benefit from being pre-generated in because of likelihood it'll be needed at some point, it would be 60, 120 or 250.
My guesses for which sizes are most commonly available (not in % of requests, but in % files):

  • the file page size: 960 + 1200/1600 (1.5x, 2x) (almost guaranteed to be rendered post-upload already, as users go confirm their work)
  • one of the common search thumbnail sizes: 60, 120, 250, 330
  • one of the more common on-page thumbnail sizes: 250, 330 + 500 (1.5x/2x) - guaranteed to be rendered as soon as it is included
  • full-screen usage - I think by this point, we're already dealing with outliers and not worth pre-generating

I noticed that I missed 2 pretty common (in terms of requests) sizes: 20 & 40px. I'm not sure where these come from, but I'm going to guess those are likely mostly (a very select subset of images used as) icons?

That said, with standardization through $wgThumbnailSteps, most of these common ones are already expected to be readily available anyway.

In the light of the above discussion, I now propose only pre-generating the one size (that the user will want to see straight after upload on commons anyway).

I'd say we should move the discussion about pre-generation to another ticket since it's a bit offtopic but in the meantime. Two things to note: pregen happens through a job which means it has a delay in order of ten seconds. When the user uploads the file, they immediately see it in the default thumbnail size (via UploadWizard) so they are actually faster than the pregen job. Another note is that there is a push to move thumbnails fully off swift and to a more cache-like infra such as ATS which we talked about it in depth in WE5&6 offsite (T345334) knowing that, I think it'll make the whole conversion about pre-gen somewhat moot since even if we pregen them, they will be deleted in 90 days. I think the focus should be on making thumbnailing faster and better. e.g. using HW acceleration if possible (T413872) and thumbnailing from an intermediate size (i.e. instead of using the original if it's larger than X, pick the 4K size and use that to thumbnail) which would make it blazing fast in really large files.

Special:NewFiles doesn't appear to be as bad as it was a few years ago, but I do think it would still benefit from pregenerating the 120px/250px thumbnails.

Change #1224051 had a related patch set uploaded (by MVernon; author: MVernon):

[operations/mediawiki-config@master] Update config to reflect new standard set of thumbnail sizes (WE 5.4.7)

https://gerrit.wikimedia.org/r/1224051

Change #1224071 had a related patch set uploaded (by MVernon; author: MVernon):

[operations/mediawiki-config@master] Stop pregenerating thumbnails

https://gerrit.wikimedia.org/r/1224071

Change #1224051 merged by jenkins-bot:

[operations/mediawiki-config@master] Update config to reflect new standard set of thumbnail sizes (WE 5.4.7)

https://gerrit.wikimedia.org/r/1224051

Mentioned in SAL (#wikimedia-operations) [2026-01-07T13:30:18Z] <ladsgroup@deploy2002> Started scap sync-world: Backport for [[gerrit:1224051|Update config to reflect new standard set of thumbnail sizes (WE 5.4.7) (T408062 T412971)]]

Mentioned in SAL (#wikimedia-operations) [2026-01-07T13:32:30Z] <ladsgroup@deploy2002> mvernon, ladsgroup: Backport for [[gerrit:1224051|Update config to reflect new standard set of thumbnail sizes (WE 5.4.7) (T408062 T412971)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2026-01-07T13:37:20Z] <ladsgroup@deploy2002> Finished scap sync-world: Backport for [[gerrit:1224051|Update config to reflect new standard set of thumbnail sizes (WE 5.4.7) (T408062 T412971)]] (duration: 07m 02s)

Change #1224071 merged by jenkins-bot:

[operations/mediawiki-config@master] Only generate 120,250 thumbs (temporary)

https://gerrit.wikimedia.org/r/1224071

Mentioned in SAL (#wikimedia-operations) [2026-01-08T17:32:59Z] <ladsgroup@deploy2002> Started scap sync-world: Backport for [[gerrit:1224071|Only generate 120,250 thumbs (temporary) (T408062 T412971)]]

Mentioned in SAL (#wikimedia-operations) [2026-01-08T17:35:41Z] <ladsgroup@deploy2002> mvernon, ladsgroup: Backport for [[gerrit:1224071|Only generate 120,250 thumbs (temporary) (T408062 T412971)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2026-01-08T17:50:32Z] <ladsgroup@deploy2002> Finished scap sync-world: Backport for [[gerrit:1224071|Only generate 120,250 thumbs (temporary) (T408062 T412971)]] (duration: 17m 34s)

Special:NewFiles doesn't appear to be as bad as it was a few years ago, but I do think it would still benefit from pregenerating the 120px/250px thumbnails.

I moved all gallery like pages to lazy loading and that has helped a lot.
(not wikitext pages however, and some people do generate pages with a bazillion images on it. Things like: https://commons.wikimedia.org/wiki/User:Atlasowa/New_video2commons/2026_January_1-10 must be a performance nightmare )

I think we're settled on this set of sizes.

Change #1237517 had a related patch set uploaded (by Ladsgroup; author: Ladsgroup):

[operations/mediawiki-config@master] MediaViewer: Adjust bucket sizes with the new thumb standard sizes

https://gerrit.wikimedia.org/r/1237517

Change #1237517 merged by jenkins-bot:

[operations/mediawiki-config@master] MediaViewer: Adjust bucket sizes with the new thumb standard sizes

https://gerrit.wikimedia.org/r/1237517

Mentioned in SAL (#wikimedia-operations) [2026-02-09T15:11:43Z] <ladsgroup@deploy2002> Started scap sync-world: Backport for [[gerrit:1237517|MediaViewer: Adjust bucket sizes with the new thumb standard sizes (T412971)]]

Mentioned in SAL (#wikimedia-operations) [2026-02-09T15:11:51Z] <ladsgroup@deploy2002> ladsgroup: Backport for [[gerrit:1237517|MediaViewer: Adjust bucket sizes with the new thumb standard sizes (T412971)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2026-02-09T15:17:46Z] <ladsgroup@deploy2002> Finished scap sync-world: Backport for [[gerrit:1237517|MediaViewer: Adjust bucket sizes with the new thumb standard sizes (T412971)]] (duration: 07m 54s)