Page MenuHomePhabricator

De-fragment thumbnail sizes in mediawiki
Open, Needs TriagePublic

Description

Currently, users can set any thumbnail sizes in mediawiki and we will serve them the exact same size. This has caused many issues: T211661: Automatically clean up unused thumbnails in Swift T345334: Cache thumbs in our caching infrastructure (e.g. ATS) and many more tickets connected to them.

A rather simple improvement to this is to expand pregen sizes (in mw) and when users set the thumbsize when editing (e.g. if they pick 110px), we pick the smallest thumbsize that's larger than the picked size (in this case: 120px). Set the image URL to that size but resize the <img> tag to the requested size (110px in this example).

For what pregen sizes we can pick, here are some data of most used sizes:

cache statussize (px)hits (in a day)
hit-front220155602049
hit-front440135746095
hit-front40111905238
hit-front16097151635
hit-front2095060013
hit-front2393020718
hit-front6082391195
hit-front96077506680
hit-front3070646190
hit-front12068306081
hit-front4553407704
hit-front33049912716
hit-front2249074909
hit-front30048777979
hit-front4448309778
hit-local22048146954
hit-front4647160522
hit-front5044400026
hit-front3543559169
hit-local44043416242
hit-front20041122192
hit-front18839760280
hit-front48033331526
hit-local16033202649
hit-local12032572116
hit-front1531546759
hit-front10030324205
hit-local33028760556
hit-front50027901900
hit-front15026586327

We don't need to change anything in swift/thumbor/ATS/varnish/etc. but it will reduce a lot of cache fragmentation and load on thumbor and storage on swift. It'll improve user experience by serving more from edge caches than backends.

This probably should be rolled out gradually and wiki by wiki (by a config flag)

Event Timeline

This seems broadly sensible - what's the concrete proposal in terms of which thumb sizes will be supported/generated?

From what I'm seeing for usages, these seems to be obvious ones:
20, 40, 60, 120, 220, 330, 440, 500, 960, (and a couple of large sizes)

From what I'm seeing for usages, these seems to be obvious ones:
20, 40, 60, 120, 220, 330, 440, 500, 960, (and a couple of large sizes)

If we were to increase the default thumb size and bundle those changes, the updated default thumbsizes will be these:

20, 40, 60, 120, 250, 330, 500, 960, (and a couple of large sizes)

Don't forget about @1.5 and @2 sizes for hidpi/retina screens

(and this might complicate vertical sizes a tad, as vertical sizes get normalised to width, which means their sizes can depend on their Aspect Ratio)

Don't forget about @1.5 and @2 sizes for hidpi/retina screens

Yeah, that's why most sizes in the suggested list are double of another size. Not sure how to tackle 1.5x though.

(and this might complicate vertical sizes a tad, as vertical sizes get normalised to width, which means their sizes can depend on their Aspect Ratio)

I'll check this.

Change #1084920 had a related patch set uploaded (by Alistair3149; author: Alistair3149):

[mediawiki/core@master] Use smallest size in wgThumbLimits for file history thumbnail

https://gerrit.wikimedia.org/r/1084920