@Koavf: "Support" where exactly? Is this only about generating thumbnails as AVIF instead of PNG of images in any kind of format? Is this about uploading files in AVIF format? Please always read and follow https://www.mediawiki.org/wiki/How_to_report_a_bug and be specific where you expect what to happen.

Additionally, support has been added by software vendors (e.g. GIMP)

I don't think so. Citations are needed for such statements.

Generating thumbnails and serving up content in MediaWiki as AVIF images. My goal is to be able to use AVIF images in the exact same way as JPEG and PNG.

Re: GIMP: https://www.zdnet.com/article/chrome-and-firefox-are-getting-support-for-the-new-avif-image-format/

Generating thumbnails and serving up content in MediaWiki as AVIF images. My goal is to be able to use AVIF images in the exact same way as JPEG and PNG.

Creating thumbnails in AVIF format may depend on the backend that you use for creating thumbnails (e.g. ImageMagick, GD, Thumbor, etc). In MediaWiki itself that would be under MediaWiki-File-management (like T27611). On Wikimedia servers that might require support in Thumbor.
Support for/allow uploading files in AVIF format in MediaWiki by default would be a different request under MediaWiki-Uploading.
And allowing uploading files in AVIF format on Wikimedia servers is another, different request under Wikimedia-Site-requests, like T27397.

@Koavf: Please make this task specific which of those is wanted (or maybe something else is wanted), or if more than one thing is wanted then please break it into clear, separate, actionable tasks.
Currently this task is not actionable because it covers way too many things. Thanks a lot!

I suspect this task tree doesn't make much sense in the order its in...

MW will need changes (at minimum) for adding AVIF to the mime map etc. An image handler subclass is probably needed too (especially if we're expecting pretty thumbnails).

And as Andre said, support (in terms of new packages, which can't just necessarily be installed straight off depending on what they are) in the thumbnailers (and the transcoders) may be needed if the current packages (or their installed versions) can't do AVIF

I'm going to mark T257718 and T257719 as stalled, as there's a lot more prep work to be done before either of those can be actioned

It probably also requires a Product type decision of whether "we" want to do this

Adding and maintaing support for an "upcoming" format isn't free. Especially as we don't know if it will recieve enough traction to become viable/widely supported

It does look like TimedMediaHandler has at least some support for https://en.wikipedia.org/wiki/AV1 as part of the WebM handler

The TimedMediaHandler support won't help here directly, except insofar as actually functionally supporting ingestion of AV1 WebM video uploads will require video transcode servers to have libaom installed, so will establish that we can install the codec library on our machines in a sensible way. :)

AVIF will require also a suitable conversion program (perhaps a backported ImageMagick, perhaps something else), and support in both MediaWiki core and thumb0r for the conversion. I've personally found adding support for things to thumb0r to be difficult as it's not easy to set up or run the regression tests.

As noted it's not a zero cost thing, and the bandwidth savings if any would have to be balanced against the disk & memory storage cost of additional file renderings, and the engineering and ops cost of getting it working.

That said it'd be lovely! But we still have serious outstanding issues with our WebP support I'd rather we fix first, especially that we DO NOT SERVE THE RIGHT FILE EXTENSION with WebP thumbnails, causing files to be saved as WebP with a ".png" extension. This really should be fixed before we add more file types into the mix.

I have experience working with Thumbor (including creating test cases), and would be willing to help implement AVIF support if it's decided that it should be a priority. However, ImageMagick only just recently got support for it through libheif 1.7.0 and libaom 2.0.0. Debian doesn't package any version of ImageMagick 7 yet and packages libheif 1.6.1 and libaom 1.0.0 in sid (unstable).

In limited local testing, I found that AVIF encoding is slow. Like, really slow.

$ time convert IMG_4610.jpg IMG_4610.avif

real	0m16.776s
user	0m49.837s
sys	0m1.337s

And that's not even a huge file by Wikimedia standards.

$ identify IMG_4610.avif 
IMG_4610.avif AVIF 4032x3024 4032x3024+0+0 8-bit YCbCr 0.000u 0:00.002

For reference:

$ time convert IMG_4610.jpg IMG_4610.tiff

real	0m0.343s
user	0m0.221s
sys	0m0.097s
$ time convert IMG_4610.jpg IMG_4610.png

real	0m4.600s
user	0m5.002s
sys	0m0.121s

Decoding is on par with other formats though.

$ time convert IMG_4610.avif IMG_4610-2.jpg 

real	0m1.128s
user	0m1.176s
sys	0m0.214s

$ ls -sh1 IMG_4610*
3.7M IMG_4610-2.jpg
2.4M IMG_4610.avif
2.6M IMG_4610.jpg
 17M IMG_4610.png
 35M IMG_4610.tiff

Considering that only 0.2 MB was saved, I don't see the point of using up 85% of the 1min maximum response time to serve images in AVIF. If file creation and editing tools gain widespread support for it, allowing AVIF uploads might be a good idea.

The issue with AVIF is that it's expensive to decode, so much so that on a lot of devices the gains in transfer time are outweighed by the extra expense in decoding time. This tradeoff is the reason why WebP was based on VP8, while VP9 was already available at the time, because VP8 was cheaper to decode. Until hardware decoding of AV1 is widely available on mobile devices, adopting AVIF thumbnails could be counter-productive for a lot of users.

It's probably useful to do some testing on low powered devices and determine the bandwidth at which the faster transfer outweighs the extra decoding time and compare that to the real life internet connectivity data we have. Doing this on a few devices could give us an idea of the percentage of visitors that would currently benefit from such a change.

But deploying this feature blindly would be risky and could result in a worsened experience for a non-negligible portion of visitors.

As it turned out, GIMP added AVIF support to version 2.10.22 in September 2020, see #2668.

This issue is stalling T257718 and T257719.
I believe that what is blocking AVIF support is generating thumbnails from AVIF files, regardless of the thumbnails' format.
I believe that AVIF is well supported by Wikimedia servers' software stack by now.
Whether AVIF is the default format for all thumbnails or whether it saves time with the processing/bandwidth trade-offs is irrelevant and should not be a blocking issue. AVIF support in general is needed for different reasons (12-bit HDR with browser support, namely).

I agree with @Trougnouf that this issue (use AVIF for thumbnails) shouldn't block progress on being able to upload AVIF files.

Using the ImageMagick 6.9.11 implementation of encoding AVIF and HEIF (libavif 0.9.3, libheif 1.12.0) is still painfully slow, using a typical large photograph (extracted PNG from a Canon CR3 raw file):

$ file IMG_0652.png
IMG_0652.png: PNG image data, 6984 x 4660, 8-bit/color RGB, non-interlaced
$ time convert IMG_0652.png IMG_0652.avif
real    0m10.608s
user    0m29.370s
sys     0m2.031s
$ time convert IMG_0652.png IMG_0652.heif
real    0m17.438s
user    0m17.694s
sys     0m0.148s
$ time convert IMG_0652.png IMG_0652.jpg
real    0m0.756s
user    0m0.648s
sys     0m0.092s

In T257652#9512007, @Jonathanischoice wrote:

I agree with @Trougnouf that this issue (use AVIF for thumbnails) shouldn't block progress on being able to upload AVIF files.

Note that this ticket isn't stalling the other mentioned ticket. Child items/tickets are blocking this ticket and those child items are in the stalling state. I think the work mentioned mostly referred to upgrading thumbor to get the new avif format supported in imagick. This is done, to those items are no longer stalled I think.

In limited local testing, I found that AVIF encoding is slow. Like, really slow.

$ time convert IMG_4610.jpg IMG_4610.avif

real	0m16.776s
user	0m49.837s
sys	0m1.337s

And that's not even a huge file by Wikimedia standards.

$ identify IMG_4610.avif 
IMG_4610.avif AVIF 4032x3024 4032x3024+0+0 8-bit YCbCr 0.000u 0:00.002

...

$ ls -sh1 IMG_4610*
3.7M IMG_4610-2.jpg
2.4M IMG_4610.avif
2.6M IMG_4610.jpg
 17M IMG_4610.png
 35M IMG_4610.tiff

Considering that only 0.2 MB was saved, I don't see the point of using up 85% of the 1min maximum response time to serve images in AVIF. If file creation and editing tools gain widespread support for it, allowing AVIF uploads might be a good idea.

In T257652#9512021, @Jonathanischoice wrote:

Using the ImageMagick 6.9.11 implementation of encoding AVIF and HEIF (libavif 0.9.3, libheif 1.12.0) is still painfully slow, using a typical large photograph (extracted PNG from a Canon CR3 raw file):

$ file IMG_0652.png
IMG_0652.png: PNG image data, 6984 x 4660, 8-bit/color RGB, non-interlaced
$ time convert IMG_0652.png IMG_0652.avif
real    0m10.608s
user    0m29.370s
sys     0m2.031s
$ time convert IMG_0652.png IMG_0652.heif
real    0m17.438s
user    0m17.694s
sys     0m0.148s
$ time convert IMG_0652.png IMG_0652.jpg
real    0m0.756s
user    0m0.648s
sys     0m0.092s

I just wanted to highlight that the two examples are not very relevant as they contain no i formation regarding the quality of the output.

Avif can achieve anywhere 20-50% of the jpeg image size given the same perceived quantity of the compressed file compared to the original.

It is true that AVIF encoding is much slower than earlier formats. If this is too CPU heavy to achieve during upload, it might be possible to use a maintenance script that generates new thumbnails in avif format during off-peak hours - perhaps even Imaginary or similar tools on other servers. The gain in bandwidth could be substantial, especially for image heavy sites like Wikimedia Commons.

Here are some examples Netflix did 2020. The examples anf comparisons are a bit down the article. https://netflixtechblog.com/avif-for-next-generation-image-coding-b1d75675fe4

In T257652#9986048, @Forza1000 wrote:

It is true that AVIF encoding is much slower than earlier formats. If this is too CPU heavy to achieve during upload, it might be possible to use a maintenance script that generates new thumbnails in avif format during off-peak hours - perhaps even Imaginary or similar tools on other servers. The gain in bandwidth could be substantial, especially for image heavy sites like Wikimedia Commons.

Here are some examples Netflix did 2020. The examples anf comparisons are a bit down the article. https://netflixtechblog.com/avif-for-next-generation-image-coding-b1d75675fe4

Most of our encoding is on the fly, as needed, cpu only. Speed is more important then total byte count for WMF I think.
This is mostly a not-enough-return-on-investment-to-be-worth-the-bother at this point I think.

Apologies - for what it's worth, using an arbitrarily chosen but reasonable quality of 80% for both conversions:

$ file IMG_0652.png
IMG_0652.png: PNG image data, 6984 x 4660, 8-bit/color RGB, non-interlaced
$ time convert -quality 80 IMG_0652.png IMG_0652.avif
real    0m30.996s
user    1m49.326s
sys     0m3.958s
$ time convert -quality 80 IMG_0652.png IMG_0652.jpeg
real    0m0.858s
user    0m0.721s
sys     0m0.136s
$ ls -l
-rw-rw-r-- 1 --- --- 1.1M Jul 17 16:19 IMG_0652.avif
-rw-rw-r-- 1 --- --- 2.4M Jul 17 16:19 IMG_0652.jpeg
-rw-rw-r-- 1 --- ---  41M Aug 24  2023 IMG_0652.png

I'm not sure that "80%" means the same thing in the different codecs, however.

In T257652#9989027, @Jonathanischoice wrote:

Apologies - for what it's worth, using an arbitrarily chosen but reasonable quality of 80% for both conversions:

No need to apologise. I am sorry that my comments came out harsh. 🙏

My intention was simply to mention that there are uses-cases where a smaller file is more important to CPU usage.

Since it is very difficult to objectively assess the quality of an image, I created a script that uses magick compare -metric ssim to mathematically compare the original with the compressed. SSIM is often used as a method to gauge the quality in video compression applications. Higher values means higher perceived quality. My idea is to be able to determine for a specific SSIM value, what the quality setting and final size each compression format has, and for a given filesize, what SSIM and quality settings were used.

I took one of the Picture of the Day images from Wikimedia Commons, converted it to PNG, resized it to the same preview sizes used on Commons. The script creates jpeg, webp and avif images using quality 10-100, in increments of 5. It then checks the SSIM and adds the result to the filenames.

Script and image gallery available at https://mirrors.tnonline.net/misc/img-compare/

That is very cool :) My only nit-pick might be that starting with a JPEG source image might bias or throw off the lossy compression, since it will already have minor block and ring artifacts (especially if you're measuring SSIM or PSNR). Might be better to find an uncompressed TIFF or RAW image. Also, it might be fun to compare HDR images with higher bits (10 or 12 per element, e.g. RAW) which is where the newer WebP and AVIF codecs do a better job; and I'd like to dig into things like PIQE or BRISQUE measurements if I get time (unlikely!) since SSIM is a bit blunt.

Unrelated, I'm not entirely convinced that ImageMagick is implementing its libavif or libheif code very well, because using the GIMP to export my example source as AVIF and HEIF takes only a couple of seconds. I'd need to dig into compile flags, documentation and command line options to see if perhaps we're missing something obvious.

In T257652#9993064, @Jonathanischoice wrote:

Also, it might be fun to compare HDR images with higher bits (10 or 12 per element, e.g. RAW) which is where the newer WebP and AVIF codecs do a better job

WebP is 8-bit only, right? (The reason I need a modern codec like AVIF/HEIC/JPEG XL is higher color bit depth.)

In T257652#9993075, @Trougnouf wrote:

WebP is 8-bit only, right? (The reason I need a modern codec like AVIF/HEIC/JPEG XL is higher color bit depth.)

Hah, so it seems - I thought they had added the VP9 intra to WebP, which supports 10 and 12 bits, but apparently not (surprised face).

I compiled libavif from https://github.com/AOMediaCodec/libavif

The results are quite interesting. avifenc has both a quality setting and a speed setting.

Using the original full sized jpeg as before:

• q40, s10: encode time 0.225s
• q40, s6: encode time 0.775s
• q40, s1: encode time 12.779s

imagemagic encode to jpeg using q40 takes 0.355s. This means we could achieve similar encoding time with avif as with jpeg, assuming we are happy with the output quality or size.

I've put the three output files in https://mirrors.tnonline.net/misc/img-compare/001_Juvenile_leopard_in_the_Serengeti_National_Park_Photo_by_Giles_Laurent/avifenc/

In the end, I think the solution in MediaWiki should be to allow custom formats for previews, thumbnails and original files. Hardcoding specific formats is likely to be suboptimal for users with divergent use-cases.

PHP 8.1 has native support for AVIF, including the speed option. WebP seems supported by all current PHP versions.

https://www.php.net/manual/en/function.imageavif.php
https://www.php.net/manual/en/function.imagewebp.php

I am now down a rabbit hole updating someone's BRISQUE, PIQE and NIQE code. It mostly now seems to work, but I don't have an NVidia card to test the BRISQUE code on.

In T257652#9997534, @Jonathanischoice wrote:

I am now down a rabbit hole updating someone's BRISQUE, PIQE and NIQE code. It mostly now seems to work, but I don't have an NVidia card to test the BRISQUE code on.

Hi. Do we have any updates regarding this ticket? 🙏

In T257652#10185741, @Forza1000 wrote:

Hi. Do we have any updates regarding this ticket? 🙏

If there are updates they will be in the ticket.

I'd just like to add that a year and a half later, using the ImageMagick 7.1.1-43 implementation in Debian Trixie, encoding AVIF is much much faster (but HEIF encoding is still slow). Using the same typical large photograph as last time (extracted PNG from a Canon CR3 raw file):

$ file IMG_0652.png
IMG_0652.png: PNG image data, 6984 x 4660, 8-bit/color RGB, non-interlaced
$ time convert IMG_0652.png IMG_0652.avif
real    0m1.653s
user    0m10.392s
sys     0m0.480s
$ time convert IMG_0652.png IMG_0652.heif
real    0m7.766s
user    1m58.705s
sys     0m0.600s
$ time convert IMG_0652.png IMG_0652.jpg
real    0m1.068s
user    0m0.926s
sys     0m0.142s