Page MenuHomePhabricator

Investigate deploying Google's Guetzli to further reduce thumbnail size of JPEG
Closed, DeclinedPublic

Description

https://arstechnica.co.uk/information-technology/2017/03/google-jpeg-guetzli-encoder-file-size/

Google has developed and open-sourced a new JPEG algorithm that reduces file size by about 35 percent—or alternatively, image quality can be significantly improved while keeping file size constant. Importantly, and unlike some of its other efforts in image compression (WebP, WebM), Google's new JPEGs are completely compatible with existing browsers, devices, photo editing apps, and the JPEG standard.

https://github.com/google/guetzli/

Event Timeline

Reedy created this task.Mar 18 2017, 7:05 PM
Restricted Application added projects: Multimedia, Commons. · View Herald TranscriptMar 18 2017, 7:05 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Pine awarded a token.Mar 18 2017, 8:14 PM
Pine rescinded a token.
Pine awarded a token.
Pine added a subscriber: Pine.
Gilles added a subscriber: Gilles.EditedMar 20 2017, 10:06 AM

I'm initially skeptical of these claims. MozJPEG had similar ones that turned out to be false when put to the test with our compression parameters.

I see a few concerning points in Google's announcement:

My hunch would be that like previous attempts such as Mozilla's, they're not generating less noise, they're generating a different kind of noise. Applying more to the color than the detail in this case. It's worth attempting, for sure, but it's hard to tell if people will universally prefer that kind of compression artifact, especially after the novelty of a different kind of compression artifact fades.

Lastly, there's a difference between the quality expectations of people pulled from the Google offices and the fidelity Commons editors expect, or the public at large does.

This will have to be tested properly, benchmarked with DSSIM tools, before making any decision about using it for all thumbnails, or possibly just thumbnails using the low quality parameter.

Peter added a subscriber: Peter.Mar 20 2017, 10:21 AM
Gilles added a comment.EditedMar 20 2017, 12:07 PM

Also, from their github page:

Guetzli uses a large amount of memory. You should provide 300MB of memory per 1MPix of the input image.

I doubt that our current image scaling hardware could deal with such a huge increase in memory consumption if we compressed all our JPGs with Guetzli. This is another aspect to take into consideration, along with render duration, since we generate a lot of thumbnails on-demand on client misses. They mention render times in minutes per megapixel, which is completely unworkable for on-demand thumbnailing.

Gilles added a comment.EditedMar 20 2017, 1:30 PM

Quick test using https://github.com/jterrace/pyssim and the "sample high quality image" Guetzli linked to on their github.

First off, guetzli doesn't provide any chroma subsampling option and generates only 4:4:4 images. This is unfortunate, because using 4:2:0 is a known optimization for smaller file size while retaining image quality. This is something we've been doing for almost a year now. Consequently, at a pyssim score as close as it gets between IM and guetzli, guetzli only wins in file size against IM's 4:4:4 image, but not its 4:2:0 image:

gilles@ubuntu:~/Documents/guetzli/bin/Release$ ./guetzli -quality 95 bees.png bees-guetzli-q95-yuv444.jpg
gilles@ubuntu:~/Documents/guetzli/bin/Release$ pyssim bees.png bees-guetzli-q95-yuv444.jpg 
0.9886096
gilles@ubuntu:~/Documents/guetzli/bin/Release$ convert -quality 93 bees.png bees-im-q93-yuv444.jpg
gilles@ubuntu:~/Documents/guetzli/bin/Release$ pyssim bees.png bees-im-q93-yuv444.jpg
0.9889951
gilles@ubuntu:~/Documents/guetzli/bin/Release$ exiftool bees-guetzli-q95-yuv444.jpg 
ExifTool Version Number         : 10.10
File Name                       : bees-guetzli-q95-yuv444.jpg
Directory                       : .
File Size                       : 37 kB
File Modification Date/Time     : 2017:03:20 14:12:56+01:00
File Access Date/Time           : 2017:03:20 14:13:13+01:00
File Inode Change Date/Time     : 2017:03:20 14:12:56+01:00
File Permissions                : rw-rw-r--
File Type                       : JPEG
File Type Extension             : jpg
MIME Type                       : image/jpeg
JFIF Version                    : 1.01
Resolution Unit                 : None
X Resolution                    : 1
Y Resolution                    : 1
Image Width                     : 444
Image Height                    : 258
Encoding Process                : Extended sequential DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Y Cb Cr Sub Sampling            : YCbCr4:4:4 (1 1)
Image Size                      : 444x258
Megapixels                      : 0.115
gilles@ubuntu:~/Documents/guetzli/bin/Release$ exiftool bees-im-q93-yuv444.jpg
ExifTool Version Number         : 10.10
File Name                       : bees-im-q93-yuv444.jpg
Directory                       : .
File Size                       : 45 kB
File Modification Date/Time     : 2017:03:20 14:14:10+01:00
File Access Date/Time           : 2017:03:20 14:14:14+01:00
File Inode Change Date/Time     : 2017:03:20 14:14:10+01:00
File Permissions                : rw-rw-r--
File Type                       : JPEG
File Type Extension             : jpg
MIME Type                       : image/jpeg
JFIF Version                    : 1.01
Resolution Unit                 : cm
X Resolution                    : 28
Y Resolution                    : 28
Image Width                     : 444
Image Height                    : 258
Encoding Process                : Baseline DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Y Cb Cr Sub Sampling            : YCbCr4:4:4 (1 1)
Image Size                      : 444x258
Megapixels                      : 0.115
gilles@ubuntu:~/Documents/guetzli/bin/Release$ convert bees.png -colorspace YUV -sampling-factor 4:2:0 -quality 93 bees-im-q93-yuv420.jpg
gilles@ubuntu:~/Documents/guetzli/bin/Release$ pyssim bees.png bees-im-q93-yuv420.jpg
0.989043
gilles@ubuntu:~/Documents/guetzli/bin/Release$ exiftool bees-im-q93-yuv420.jpg
ExifTool Version Number         : 10.10
File Name                       : bees-im-q93-yuv420.jpg
Directory                       : .
File Size                       : 33 kB
File Modification Date/Time     : 2017:03:20 14:16:49+01:00
File Access Date/Time           : 2017:03:20 14:17:08+01:00
File Inode Change Date/Time     : 2017:03:20 14:16:49+01:00
File Permissions                : rw-rw-r--
File Type                       : JPEG
File Type Extension             : jpg
MIME Type                       : image/jpeg
JFIF Version                    : 1.01
Resolution Unit                 : cm
X Resolution                    : 28
Y Resolution                    : 28
Image Width                     : 444
Image Height                    : 258
Encoding Process                : Baseline DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Y Cb Cr Sub Sampling            : YCbCr4:2:0 (2 2)
Image Size                      : 444x258
Megapixels                      : 0.115

Tl;dr; at equal SSIM score, guezli's 4:4:4 jpeg is 37kb, IM's 4:4:4 jpeg is 45kb and IM's 4:2:0 jpeg is 33kb.

The main author of Guetzli seems oddly unfamiliar with chroma subsampling. But it's possible than if/when the option is added, guetzli would outperform libjpeg. With that option being currently missing, there's no reason to adopt Guetzli for our thumbnailing, IMHO. We could run our own wider scale double-blind human evaluation of Guetzli, but the initial SSIM scores don't make it look like it would be a worthwhile investment.

And that's still ignoring the awful runtime and memory consumption performance...

Reedy triaged this task as Low priority.Mar 20 2017, 1:31 PM

I doubt that our current image scaling hardware could deal with such a huge increase in memory consumption if we compressed all our JPGs with Guetzli. This is another aspect to take into consideration, along with render duration, since we generate a lot of thumbnails on-demand on client misses. They mention render times in minutes per megapixel, which is completely unworkable for on-demand thumbnailing.

I suspect optimising them at a later date (via job queue or similar) just makes the problem worse; "hey, you've downloaded it once. but that's out of date now. Get this newer smaller version!"

Certainly, the end goal is nice, if the gains are to be believed, and the visual differences are acceptable. The memory tradeoff can be dealt with, the time for processing not so much. People aren't going to wait for minutes for thumbnails

Thanks for doing a bit more thinking/research on it

Going to triage this as Low, could potentially be Lowest.

We've talked about doing a later more expensive pass to optimize images further for other file formats (PNG, I believe), but it's a project of its own. And I think we have yet to stumble into gains big enough that would apply to a large enough chunk of our content that it would be something that gets picked up as a project.

In the case of Guetzli, though, I can't even make it render a smaller file at equal quality than what we currently make with IM, because of Guetzli's lack of chroma subsampling option. I think it's worth keeping an eye on it and revisiting when they add an option to generate 4:2:0 jpegs.

Reedy changed the task status from Open to Stalled.Mar 20 2017, 1:40 PM
Reedy added a project: Upstream.

Do this for now then? :)

Based on their explanations, they seem to be trading noise in details for noise in color.
[..] No mention of whether they're comparing 4:4:4, 4:2:0 JPEGs, etc.
My hunch would be that like previous attempts such as Mozilla's, they're not generating less noise, they're generating a different kind of noise. Applying more to the color than the detail in this case.

It seems Guetzli is optimised for high-quality photographs (80-95% quality JPEG, 4:4:4 colors) - e.g. when using large images as background images for a site, or when you have lots images on the page that aren't semantically in the role of thumbnail but rather the content itself, like on Google Photos or Flickr.

Given we use a <80% JPEG quality and already have 4:2:0 set as reduced color scale, it seems Guetzli optimises in is an area not applicable to us. Of course, if it weren't for the high memory usage and the fact that it takes minutes to generate, we could consider it as a way of achieving more quality at the same file size. But given that it does have high memory requirements and long run-time, it seems primarily intended when publishing can be delayed. E.g. when generating a static website, or if the entity in question has a more controlled publication time (e.g. social media sites could in theory delay publication in a more controlled way - the uploader could be shown a high-quality thumbnail of larger file size until that time).

Tgr added a subscriber: Tgr.Mar 20 2017, 7:18 PM

It seems Guetzli is optimised for high-quality photographs (80-95% quality JPEG, 4:4:4 colors) - e.g. when using large images as background images for a site, or when you have lots images on the page that aren't semantically in the role of thumbnail but rather the content itself, like on Google Photos or Flickr.

Wikidata-Page-Banner comes to mind.

I don't think there's ever a case where it's useful for us to serve 4:4:4 jpegs. Even if Guetzli makes them "less expensive", at equal quality in my small test above it's still 9% bigger for absolutely no objective higher quality than a 4:2:0 quality equivalent. And that's at 95 quality in Guetzli on the reference image they provide, which is their advertised objective. So even for what it intends to do, it falls short compared to 4:2:0 images on objectively-measured visual differences (which includes color).

Their methodology is all wrong, they set out to do something and validated it through a very shoddy survey, ignoring the existing objective benchmarks. If they think that SSIM is an inappropriate benchmark for what they're trying to do, they should have explained why. But the lack of mention of SSIM, the fact that the main author doesn't seem to know what 4:2:0 chroma subsampling is, indicate to me that they probably made those great claims prematurely and without due diligence. And this probably wasn't built for the web. We can only speculate what they needed it for.

Gilles moved this task from Inbox to Radar on the Performance-Team board.Mar 23 2017, 9:01 PM
Krinkle closed this task as Declined.Jun 27 2017, 3:38 AM

Closing un-actionable task that doesn't (and can't) track any particular upstream action. If and when Guetzli becomes relevant for 4-2-0 thumbnails or the web in general, we'll find out and possibly revisit this.