We put that off before because we were waiting to have time to work on rectangular buckets, but this is having a performance impact right now that could be easily fixed by collapsing some of the buckets together.
By being infrequently accessed, some of the buckets are increasing the likelihood of thumbnails having to be pulled from Swift instead of being found in Varnish. Pulling from Swift makes for requests that are 3 times slower on average. Meaning that we can afford to serve slightly larger images, for example, if it greatly increases their likelihood of being served from Varnish.
Here are some stats about which bucket sizes are most encountered in Varnish misses that need to pull the thumbnail from Swift:
SELECT EXP(AVG(LOG(event_contentLength))) AS avgsize, COUNT(*) AS count, event_imageWidth FROM MultimediaViewerNetworkPerformance_11030254 WHERE event_type = 'image' AND event_varnish1hits = 0 and event_varnish2hits = 0 AND event_varnish3hits = 0 AND event_timestamp - event_lastModified > 60 GROUP BY event_imageWidth ORDER BY count DESC
177045.0831227364 12005 1024
115380.94911617086 10328 640
127670.17162169864 9549 800
243925.1228200158 9076 1280
463829.1836617856 4880 1920
43604.204761961795 1325 320
679914.8144343277 973 2560
734863.5032724637 217 2880
The least controversial merges are thus probably:
640 & 800 => would only serve images that are 11% larger on average for hits that would have gone to 640.
1024 & 1280 => images that would have hit the 1024 bucket will be 38% larger on average, but 1024 is the biggest offender in sizes hitting Swift rather than Varnish
I propose to merge 640 & 800 first and to measure the exact gain. Then we'll be able to assess if merging 1024 & 1280 will be a net gain or not (merging them doesn't mean that they will always hit varnish, meaning that the 3-fold perf advantage will be lower in practice).