• Book of Speed: A common understanding is that progressive formats produce larger files. While this seems to be true for GIF and PNG, it doesn't look so clear-cut for JPEGs. In a study of over 10000 JPEGs from all over the web, I found that results vary from one image to the next. But plotting a trendline (Figure 5.3.) shows that images of file size 10K and over have a better chance of being smaller when using the progressive JPEG format. The difference could be up to 10% (with 3% median) but in extreme cases that translated to 250K of savings.
• Lots of WebPagetest links of real-world websites loaded with or without interlacing. Also lots of interesting discussion in the comments.
• Analysis of PNG interlacing. The difference bitween the interlaced preview and the original seems barely perceptible by the time 25% of the file has been downloaded.
• biometric research that claims that interlaced JPEG rendering actually makes users more unhappy. (Somewhat questionable as it is a not-too-subtle advertisement for their alternative rendering technology.)

jpegtran can convert JPEG files to interlaced and we already run it on JPEGs so adding interlacing should be pretty trivial. imagemagick apparently supports PNG / GIF interlacing.

Some potential problems:

• size overhead (as the above article points out this is typically not the case for JPEGs but GIF / PNG overhead is claimed to be larger)
• users disliking the "blurred" look (this was a frequent complaint with MediaViewer which used a somewhat similar trick, by displaying the thumbnail in the article enlarged and with a blur filter while the original was loading)
• some CPU overhead on the client
• time/memory overhead on the rendering server. For large files this could be significant. (E.g. for PNG interlacing (Adam7) seven consecutive processing passes need to be done.) Could be mitigated by making sure that interlacing is not applied above a size limit.

Hi @Tgr. Please associate at least one project with this task, otherwise nobody can find this task when searching in the corresponding project(s). Thanks.

time/memory overhead on the rendering server. For large files this could be significant. (E.g. for PNG interlacing (Adam7) seven consecutive processing passes need to be done.) Could be mitigated by making sure that interlacing is not applied above a size limit.

Also, if we ever have to resize the file, interlaced jpegs take much more memory to resize then non-interlaced.

In T120032#1846006, @Bawolff wrote:

Also, if we ever have to resize the file, interlaced jpegs take much more memory to resize then non-interlaced.

As in, resize on the client-side?

In T120032#1846065, @Tgr wrote:

In T120032#1846006, @Bawolff wrote:

Also, if we ever have to resize the file, interlaced jpegs take much more memory to resize then non-interlaced.

As in, resize on the client-side?

I meant server. e.g. If we ever do resize chaining. I'm not sure about the behaviour on the client side, but it wouldn't surprise me as presumably both image magick and web browsers use libjpeg in the same way (although generally we don't resize on client, or at least not by more than a factor of 2, so not that much of an issue).

In T120032#1846070, @Bawolff wrote:

I meant server. e.g. If we ever do resize chaining.

We would just have to make sure that it's only done in the last step. That would be needed anyway for destructive transformations like sharpening or aggressive JPEG compression.

I'm not sure about the behaviour on the client side, but it wouldn't surprise me as presumably both image magick and web browsers use libjpeg in the same way (although generally we don't resize on client, or at least not by more than a factor of 2, so not that much of an issue).

We do a lot of resizing with a near-1 factor (MediaViewer and the mobile apps bucket image sizes for better caching, and IIRC MediaWiki thumbnails are snapped to 5px) but then I imagine browser have to keep the whole image in memory anyway.

Change 264048 had a related patch set uploaded (by Unicornisaurous):
Add support for image interlacing of Bitmap type images

https://gerrit.wikimedia.org/r/264048

Sure, it's always easier to ignore a task which has a project than to find one which doesn't.

Change 264435 had a related patch set uploaded (by Unicornisaurous):
Add support for image interlacing to VipsScaler extension

https://gerrit.wikimedia.org/r/264435

So what do I need to do next, now that theres a working patch?

@Tgr: I think you mentioned that I should work with ops to figure out what the max interlacing size should be set to by default on WMF wikis? What would be the first step? Pinging someone on phabricator, asking on IRC, or something else?

Change 264048 merged by jenkins-bot:
Add support for image interlacing of Bitmap type images

https://gerrit.wikimedia.org/r/264048

Change 264435 merged by jenkins-bot:
Add support for image interlacing in the VipsScaler extension

https://gerrit.wikimedia.org/r/264435

Well, first step is to get the code merged, since I wanted to leave it around a while in case anyone else wants to review it, and then totally forgot about it :/ I merged it now, which given our deploy cadence means that it will be live next Wednesday.

The next step is to figure out what test is needed to come up with safe size limits for using interlacing, and do it. Probably involves writing a script to render a bunch of large interlaced thumbnails and watch server metrics, but I'm not a good person to help you with that. You can ask @faidon, he can at least direct you to the right person to talk with. If you can find some facts on the internet about the resource consumption of imagemagick and vips (normal vs. interlaced) and collect them here, that will probably make things faster.

You'll probably need to know which images are handled by vips and which by imagemagick. Wikimedia's configuration is here; CommonSettings.php is the file that sets up the VipsScaler rules.

After figuring out how to test, you'll need to raise the size limit (as interlacing is now completely disabled). You can find a description of how configuration files are arranged here and the process by which you can change them here.

Our documentation is often not that great so feel always free to ask me or others on IRC instead of spending too much on figuring things out from those texts :)

This is another cool way of showing how images are shown when they are interlaced compared to non interlaced: https://github.com/pmeenan/progressive-scans

Functionally this is done; @Unicornisaurous did all the non-configuration work needed. Then I switched teams and completely forgot about this. Sorry :(

For enabling in Wikimedia, which was the main motivation, T120032#1973426 is still accurate: someone would have to test interlacing (maybe on beta, maybe in production) to see what are safe limits to use in production. After that we'd have to figure out whether it actually improves reading experience. We don't have a Multimedia team anymore; maybe someone from the Web team or the Commons technical contributor community is interested in picking this up? I still think this would be a good thing to try, but all my volunteer time is tied down in other projects.

Thumbnail processing on Wikimedia wikis is now handled by Thumbor and not MediaWiki. @Unicornisaurous's patches will still be helpful, but they will have to be ported from PHP to Python. It would probably make sense to do this in a Thumbor filter, like we do for conditional sharpening. Thumbor actually has built-in support for progressive JPGs, but it's disabled in our configuration.

In T120032#5772913, @Tgr wrote:

Functionally this is done; @Unicornisaurous did all the non-configuration work needed. Then I switched teams and completely forgot about this. Sorry :(

For enabling in Wikimedia, which was the main motivation, T120032#1973426 is still accurate: someone would have to test interlacing (maybe on beta, maybe in production) to see what are safe limits to use in production. After that we'd have to figure out whether it actually improves reading experience. We don't have a Multimedia team anymore; maybe someone from the Web team or the Commons technical contributor community is interested in picking this up? I still think this would be a good thing to try, but all my volunteer time is tied down in other projects.

@Tgr could you create an explicit task detailing what you'd need readers web to do? I wasn't really able to get a sense of what's left here. For example is this a SWAT deploy at this point? I don't have much knowledge of this subject but happy to help you champion it if you think it's important. Also wondering if this might overlap with the structured data team's goals.

Basically, the patches:

• check whether the thumbnail has an interlace- parameter (ie. instead of the thumbnail file name being 500px-SomeFile.png it would be interlace-500px-SomeFile.png)
• if it has, and and the file type is handled via VipsScaler, it adds vips jpegsave --interlace (or pngsave) to the chain of VipsScaler commands
• otherwise if the file is smaller than $wgMaxInterlacingAreas[<mimetype>] pixels, it appends -interlace JPEG or -interlace PNG or -interlace GIF (the last one only on ImageMagick 6.3.4+) to the ImageMagick command arguments.

This would have to be replicated in Thumbor.