Page MenuHomePhabricator

Make image thumbnails interlaced whenever possible
Open, LowPublic

Description

Interlaced [1] thumbnails offer a better viewing experience, especially on slow connections, as a slightly lower-quality but recognizable version of the image is displayed much sooner in the loading process.

Demonstration.


See also:

[1] Often also called progressive loading; but the ability to render an image while loading, in a non-interlaced, top-to-bottom way, is also often called progressive loading, so that terminology can be confusing.

Event Timeline

Tgr created this task.Dec 1 2015, 11:09 PM
Tgr raised the priority of this task from to Needs Triage.
Tgr updated the task description. (Show Details)
Tgr added a subscriber: Tgr.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptDec 1 2015, 11:09 PM
Tgr added a comment.EditedDec 1 2015, 11:33 PM
  • Book of Speed: A common understanding is that progressive formats produce larger files. While this seems to be true for GIF and PNG, it doesn't look so clear-cut for JPEGs. In a study of over 10000 JPEGs from all over the web, I found that results vary from one image to the next. But plotting a trendline (Figure 5.3.) shows that images of file size 10K and over have a better chance of being smaller when using the progressive JPEG format. The difference could be up to 10% (with 3% median) but in extreme cases that translated to 250K of savings.
  • Lots of WebPagetest links of real-world websites loaded with or without interlacing. Also lots of interesting discussion in the comments.
  • Analysis of PNG interlacing. The difference bitween the interlaced preview and the original seems barely perceptible by the time 25% of the file has been downloaded.
  • biometric research that claims that interlaced JPEG rendering actually makes users more unhappy. (Somewhat questionable as it is a not-too-subtle advertisement for their alternative rendering technology.)
Tgr added a comment.Dec 1 2015, 11:46 PM

jpegtran can convert JPEG files to interlaced and we already run it on JPEGs so adding interlacing should be pretty trivial. imagemagick apparently supports PNG / GIF interlacing.

Tgr added a comment.Dec 2 2015, 12:10 AM

Some potential problems:

  • size overhead (as the above article points out this is typically not the case for JPEGs but GIF / PNG overhead is claimed to be larger)
  • users disliking the "blurred" look (this was a frequent complaint with MediaViewer which used a somewhat similar trick, by displaying the thumbnail in the article enlarged and with a blur filter while the original was loading)
  • some CPU overhead on the client
  • time/memory overhead on the rendering server. For large files this could be significant. (E.g. for PNG interlacing (Adam7) seven consecutive processing passes need to be done.) Could be mitigated by making sure that interlacing is not applied above a size limit.

Hi @Tgr. Please associate at least one project with this task, otherwise nobody can find this task when searching in the corresponding project(s). Thanks.

Restricted Application added projects: Multimedia, Commons. · View Herald TranscriptDec 2 2015, 7:15 PM
Restricted Application added a subscriber: Steinsplitter. · View Herald Transcript
Bawolff added a subscriber: Bawolff.Dec 2 2015, 9:41 PM

time/memory overhead on the rendering server. For large files this could be significant. (E.g. for PNG interlacing (Adam7) seven consecutive processing passes need to be done.) Could be mitigated by making sure that interlacing is not applied above a size limit.

Also, if we ever have to resize the file, interlaced jpegs take much more memory to resize then non-interlaced.

Restricted Application added a subscriber: Matanya. · View Herald TranscriptDec 2 2015, 9:41 PM
Tgr added a comment.Dec 2 2015, 9:56 PM

Also, if we ever have to resize the file, interlaced jpegs take much more memory to resize then non-interlaced.

As in, resize on the client-side?

Also, if we ever have to resize the file, interlaced jpegs take much more memory to resize then non-interlaced.

As in, resize on the client-side?

I meant server. e.g. If we ever do resize chaining. I'm not sure about the behaviour on the client side, but it wouldn't surprise me as presumably both image magick and web browsers use libjpeg in the same way (although generally we don't resize on client, or at least not by more than a factor of 2, so not that much of an issue).

Tgr added a comment.Dec 2 2015, 10:08 PM

I meant server. e.g. If we ever do resize chaining.

We would just have to make sure that it's only done in the last step. That would be needed anyway for destructive transformations like sharpening or aggressive JPEG compression.

I'm not sure about the behaviour on the client side, but it wouldn't surprise me as presumably both image magick and web browsers use libjpeg in the same way (although generally we don't resize on client, or at least not by more than a factor of 2, so not that much of an issue).

We do a lot of resizing with a near-1 factor (MediaViewer and the mobile apps bucket image sizes for better caching, and IIRC MediaWiki thumbnails are snapped to 5px) but then I imagine browser have to keep the whole image in memory anyway.

MarkTraceur triaged this task as Low priority.Dec 21 2015, 8:52 PM
MarkTraceur added a subscriber: MarkTraceur.

Change 264048 had a related patch set uploaded (by Unicornisaurous):
Add support for image interlacing of Bitmap type images

https://gerrit.wikimedia.org/r/264048

I'm working on some if not all of this task for GCI under the mentorship of @Tgr. (Should this task be on the Google-Code-In-2015 project, since it's already imported and published? However, it is a multi-part task.)

Sure, it's always easier to ignore a task which has a project than to find one which doesn't.

Change 264435 had a related patch set uploaded (by Unicornisaurous):
Add support for image interlacing to VipsScaler extension

https://gerrit.wikimedia.org/r/264435

So what do I need to do next, now that theres a working patch?

@Tgr: I think you mentioned that I should work with ops to figure out what the max interlacing size should be set to by default on WMF wikis? What would be the first step? Pinging someone on phabricator, asking on IRC, or something else?

Change 264048 merged by jenkins-bot:
Add support for image interlacing of Bitmap type images

https://gerrit.wikimedia.org/r/264048

Change 264435 merged by jenkins-bot:
Add support for image interlacing in the VipsScaler extension

https://gerrit.wikimedia.org/r/264435

Tgr added a subscriber: faidon.Jan 28 2016, 9:16 AM

Well, first step is to get the code merged, since I wanted to leave it around a while in case anyone else wants to review it, and then totally forgot about it :/ I merged it now, which given our deploy cadence means that it will be live next Wednesday.

The next step is to figure out what test is needed to come up with safe size limits for using interlacing, and do it. Probably involves writing a script to render a bunch of large interlaced thumbnails and watch server metrics, but I'm not a good person to help you with that. You can ask @faidon, he can at least direct you to the right person to talk with. If you can find some facts on the internet about the resource consumption of imagemagick and vips (normal vs. interlaced) and collect them here, that will probably make things faster.

You'll probably need to know which images are handled by vips and which by imagemagick. Wikimedia's configuration is here; CommonSettings.php is the file that sets up the VipsScaler rules.

After figuring out how to test, you'll need to raise the size limit (as interlacing is now completely disabled). You can find a description of how configuration files are arranged here and the process by which you can change them here.

Our documentation is often not that great so feel always free to ask me or others on IRC instead of spending too much on figuring things out from those texts :)

This is another cool way of showing how images are shown when they are interlaced compared to non interlaced: https://github.com/pmeenan/progressive-scans

MarkTraceur moved this task from Untriaged to Triaged on the Multimedia board.Dec 6 2016, 3:49 PM
Restricted Application added a subscriber: Poyekhali. · View Herald TranscriptDec 6 2016, 3:49 PM