Page MenuHomePhabricator

Create thumbnail for animated WebP files
Open, Needs TriagePublicFeature

Description

Hello!

Category:Animated_WebP_files have no thumbnails. I found no task asking for it to be fixed so I created this.

Event Timeline

Aklapper changed the subtype of this task from "Task" to "Feature Request".

@Jonteemil: Thanks for reporting this. For future reference, please use the feature request form (linked from the top of the task creation page) to create feature requests. Thanks.

We have four options:

  1. Treat them like media: thumbnail to a static png/webp, and make TimedMediaHandler deal with the rest of it. That would mean no looping & no inline autoplay.
  2. Treat them like animated SVGs: thumbnail to a static png/webp and forget about the animation.
  3. Treat them like GIF: thumbnail to a gif, potentially serving awebp if the client supports it (currently not done T290351)
  4. Thumbnail to awebp only, clients not supporting it get nothing.

Notwithstanding the various tasks to treat GIF as timed media (which have been collecting dust for years), #3 or #4 are probably the best options. #3 would mean usually thumbnailing to a worse format, but #4 would depend on browser support (at least in Grade A but preferably also Grade C as well). There is also a sort of option 3.5 of serving awebp unless the client doesn't support it, then falling back to gif or png. We don't quite do that for any other file though, so it could require more work to implement.

Currently we use gifsicle to thumbnail gifs, which only supports reading and writing gifs. We use cwebp for writing webps, which doesn't support animated webp. However, webpmux, which is packaged alongside cwebp, can work with animated webp.

I tried resizing an animated webp with imagemagick on my laptop, and it did work. However, I'm running a considerably newer version compared to production Thumbor.

$ convert -version
Version: ImageMagick 7.1.0-5 Q16 x86_64 2021-08-22 https://imagemagick.org
Copyright: (C) 1999-2021 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC HDRI Modules OpenMP(4.5) 
Delegates (built-in): bzlib cairo djvu fontconfig freetype heic jbig jng jp2 jpeg lcms lqr ltdl lzma openexr pangocairo png raqm raw rsvg tiff webp wmf x xml zip zlib
$ cwebp -version
1.2.1

With ImageMagick 6.9.7-4, it just tries to use cwebp to open the animated image and fails.

$ convert --resize 220x file.webp 220px-file.webp
Error! Decoding of an animated WebP file is not supported.
       Use webpmux to extract the individual frames or
       vwebp to view this image.
Decoding of /tmp/magick-12WzNO-IXVTG78 failed.
Status: 4(UNSUPPORTED_FEATURE)
convert-im6.q16: unable to open image `--resize': No such file or directory @ error/blob.c/OpenBlob/2701.
convert-im6.q16: no decode delegate for this image format `' @ error/constitute.c/ReadImage/504.
convert-im6.q16: unable to open image `220x': No such file or directory @ error/blob.c/OpenBlob/2701.
convert-im6.q16: no decode delegate for this image format `' @ error/constitute.c/ReadImage/504.
convert-im6.q16: delegate failed `'dwebp' -pam '%i' -o '%o'' @ error/delegate.c/InvokeDelegate/1949.
convert-im6.q16: unable to open file `/tmp/magick-121td0HBb9wtim': No such file or directory @ error/constitute.c/ReadImage/544.
convert-im6.q16: no images defined `220px-file.webp' @ error/convert.c/ConvertImageCommand/3258.
$ convert -version
Version: ImageMagick 6.9.7-4 Q16 x86_64 20170114 http://www.imagemagick.org
Copyright: © 1999-2017 ImageMagick Studio LLC
License: http://www.imagemagick.org/script/license.php
Features: Cipher DPC Modules OpenMP 
Delegates (built-in): bzlib djvu fftw fontconfig freetype jbig jng jp2 jpeg lcms lqr ltdl lzma openexr pangocairo png tiff wmf x xml zlib

I'm not sure exactly when imagemagick started supporting animated webp, and I don't know when Debian started compiling with the webp delegate included. Based on this it should work in Buster. I'll have to test a bit more on Buster, but for now I'm calling it blocked on T216815: Upgrade Thumbor to Buster.

If we directly thumbnail to awebp, we'd probably want to think about if there should be an equivalent to MAX_ANIMATED_GIF_AREA for awebp. Thumbnailing to gif would include that restriction.

For Animated PNG, we render to a still frame png when we thumbnail. Note that this is primarily a side effect of the pipeline if i'm not mistaken. It was never much of an active decision.

For Animated PNG, we render to a still frame png when we thumbnail. Note that this is primarily a side effect of the pipeline if i'm not mistaken. It was never much of an active decision.

Yeah, animated PNGs and SVGs not animating in thumbnails is a result of a lack of support for apng in ImageMagick and CSS animation in librsvg, respectively. I did find libraries that can convert apng to gif (or webp), so we could make it work by converting to a different format first. Only other way would be to explode the file into individual frames, resize each, then put it back together again.

Not sure which is worse: thumbnailing an apng to a gif or a static png.

For what it's worth, there is desire for this outside of WMF, on a site not using Thumbor (that's currently using 1.23 with WebP support backported from 1.26, but let's assume it'll be upgraded soon). Personally my suggestion would be to thumbnail to an animated webp, since then it'd be a viable alternative for animations that aren't suitable for GIF or video treatment, and support has increased. Perhaps offering a fallback to a static PNG via <picture> with the webp in <source> would be icing on the cake, though.

Regarding WebP on Debian - the version in Buster is not ideal (though at least it's secure now, after some poking). Not sure if using the version from sid is feasible, but if so it might be a better choice. Or encouraging/facilitating a backport of the newer version.

As of ImageMagick 7.0.10-31, Animated PNG is supported

Note, you must use an explicit image format specifier to read an APNG (apng:myImage.apng) image sequence, otherwise it assumes a PNG image and only reads the first frame.

I haven't tested this yet, but someone could verify that this is indeed how to do it, and check if it is needed on both input and/or output and then adapt the converter.

Kizule subscribed.

I'll try to work on this.

I'll try to work on this.

you can tag me for review

Change #1270030 had a related patch set uploaded (by Zoranzoki21; author: Zoranzoki21):

[mediawiki/core@master] Add support for animated WebP files

https://gerrit.wikimedia.org/r/1270030

I'm starting to get somewhere:

Without my patchWith my patch
image.png (975×707 px, 256 KB)
image.png (980×867 px, 194 KB)

It might be that something is missing at Patchdemo, will see. This is still WIP.

Test wiki created on Patch demo by Kizule using patch(es) linked to this task:
https://20a6771106.catalyst.wmcloud.org/w/

Test wiki on Patch demo by Kizule using patch(es) linked to this task was deleted:

https://patchdemo.wmcloud.org/wikis/fc9191f562/w/

Test wiki on Patch demo by Kizule using patch(es) linked to this task was deleted:

https://20a6771106.catalyst.wmcloud.org/w/

I'm starting to get somewhere:

Without my patchWith my patch
image.png (975×707 px, 256 KB)
image.png (980×867 px, 194 KB)

It might be that something is missing at Patchdemo, will see. This is still WIP.

It shows a preview now, which is still not animated, but MediaViewer shows the file as animated. https://patchdemo.wmcloud.org/wikis/055e96e2b6/wiki/File:Arc_2V_16kA_disjoncteur_ultra_rapide_courant_continu_(dessus).webp

Without the patch (or on Commons), the thumbnail is not shown at all.

I'll try to work on this.

you can tag me for review

Tagged.

Without the patch (or on Commons), the thumbnail is not shown at all.

Yes, that's what the canRender tells it. "i do not support making a thumbnail for this". You also changed it to always output a webp instead of a png. I don't think that is needed for this (and it has a ticket of its own T338342). i'd just make the thumbnail format conditional on being animated or not at first.

It shows a preview now, which is still not animated, but MediaViewer shows the file as animated.

Correct. it does not generate animated webp thumbnails. MMV works because it is using the original, as it is a relative small dimension and will thus use the original file.

I see you have a bunch of changes to the render functions in BitmapHandler.php. These are correct, but of note is that they are not what Wikimedia Foundation actually uses to generate thumbnails. The foundation uses thumbor. For that you will have to adapt the thumbor service as well:

https://wikitech.wikimedia.org/wiki/Thumbor
Specifically https://gerrit.wikimedia.org/r/plugins/gitiles/operations/software/thumbor-plugins/+/refs/heads/master/wikimedia_thumbor/engine/imagemagick/imagemagick.py

This then also needs to be separately deployed (and it's unfortunately not available in patchdemo).

Change #1270150 had a related patch set uploaded (by TheDJ; author: TheDJ):

[operations/software/thumbor-plugins@master] Supported animated webp thumbnailing

https://gerrit.wikimedia.org/r/1270150

Would it be helpful to split thumbor and its deployment into a separate task or tasks? Or alternatively (since this issue was created with Commons in mind) to split the changes to base MediaWiki, so that these can potentially make their way into a release prior to WMF's own deployment of the feature?

From the above comment it sounds like the bitmap handling code in question wouldn't be used on WMF anyway (except potentially in some case where thumbor is unavailable?) and so it might not be an issue for it to be released while thumbor and any associated review processes remain pending.

(While typing this up it seems like TheDJ has had a stab at the thumbor stuff, which is awesome, but the question of review/deploy still stands - I don't know the policy in this area or how long it might take, but I imagine it's not instant.)

@Kizule In my mind I had already merged the metadata part of this, but it seems I did not. I will give it a final pass later this week,