| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Stalled | None | T209437 Migrate primary Wikimedia video format from VP9 to AV1 | |||
| Resolved | TheDJ | T209440 Add support for AV1 as an ingestion and transcoding target format |
Event Timeline
Note that this is dependent on suitable playback support being widespread. While on desktops, Chrome and Firefox are shipping AV1 now or in next versions, and Microsoft is previewing it for Edge, we don't yet know the situation for Safari. I've started adding AV1 decode to ogv.js, but it's slower than decoding VP9 so far so may require running lower resolutions (unless WebAssembly threading arrives soon on Safari).
For mobile, currently no phones have AV1 hardware support, though Android Chrome may get it in software. Eventually it's expected to be more widespread, at least as much as VP9 is today, but this is not something we can be too aggressive on.
It's hard to say when the right time is, but waiting for more widespread support is a good idea. Imagine we start transcoding now and when hardware implementations finally come around, nothing plays because their implementation is quirky.
Also, we should compare VP9 and AV1 ourselves. I trust few third party evalutations and those that I do trust are absolutely not Google, some university and Facebook.
And who am I to say that? Well, I've encoded movies with DivX 3.11. 'Nuff said.
As a follow-up to @AlexisJazz and @brion's comments above, AV1 hardware decoding support is very limited right now - see en-wiki:AV1#Hardware for some notes. Even VP9 and HEVC/x265 hardware decoding is relatively new ... NVidia has a page on their dev site that talks about hardware support. Onboard Intel graphics chipsets aren't much better - yes, the newest models can typically handle it, but even 3-4 years old (or the lower-end low-power stuff) might not.
Speaking from experience trying to watch videos on YouTube served up (by default) in VP9 on a computer that doesn't have H/W decoding for VP9 (but does for avc1/h.264), it's pretty much impossible to comfortably watch a video higher than 480p resolution (not to mention the CPU pegging at 100% and the associated battery drain). Switching to h.264/mp4 and the H/W decoder kicks in -- even 4k -- no problems.
Is it possible to detect H/W decoding capabilities in the browser? (I'm thinking out-loud -- I've never checked and just now thought about it.) It would be interesting to try and figure out the current capability set that's out there... and use that data as a guide on when to switch? (I'm guessing the answer is no, because it would seem like YouTube would be able to detect that and automatically serve up whichever format is hardware-decoding supported on the user's machine - it doesn't appear to do that... but that's assuming that they would do that. :) )
Quick edit: Yes, there's an API for that ;) - not supported on all browsers, but modern ones, yep! https://developer.mozilla.org/en-US/docs/Web/API/Media_Capabilities_API
Native CPU-run VP9 seems pretty ok through 720p on my lower-spec machines, but YMMV. :) AV1 is much more CPU-intensive and I definitely don't recommend switching to it wholesale yet, especially at high resolutions.
Use of Media Capabilities to pick a preferred format is a definite possibility, for supporting browsers. (It's still new so not everything has it reliably, IIRC, but it should cover current Chrome and Firefox and that's most of the world.)
(Note that the reason we're not on h.264 is the patent licensing requirement; until that gets resolved -- either by waiting until 2028 for the patents to expire or overturning the 2014 Commons RFC and reinterpreting or changing the foundation's open file formats policy -- we are stuck working around it with other formats.)
Alternate possibilities for generating files also include generating AV1 only for low resolutions (where the bitrate savings make the most difference and the CPU cost makes the least), which I believe was YouTube's first step on deploying AV1. Anyway no hurry, we don't have to be an early adopter on this when we have such limited resources to apply for now...
Significant Wikimedia community support requested for Wikimedia Commons video management
petition
discussion thread
YouTube would be able to detect that and automatically serve up whichever format is hardware-decoding supported on the user's machine - it doesn't appear to do that... but that's assuming that they would do that. :) )
YouTube is Google, so they have an incentive to push VP9/AV1.
It would be interesting to try and figure out the current capability set that's out there..
There's also https://store.steampowered.com/hwsurvey which is obviously limited to Steam users and desktop, but can provide some level of insight anyway.
(Note that the reason we're not on h.264 is the patent licensing requirement; until that gets resolved -- either by waiting until 2028 for the patents to expire or overturning the 2014 Commons RFC and reinterpreting or changing the foundation's open file formats policy -- we are stuck working around it with other formats.)
Based on https://meta.wikimedia.org/wiki/Have_the_patents_for_MPEG-4_Visual_expired_yet%3F I'm pretty sure MPEG-4 Visual (which is NOT h.264) should be fair game by now. (but we should ask legal to be certain) The Philips patent should be irrelevant (likely optional to implement in decoders and decoders are the user's problem), but any MPEG-4 rollout wouldn't happen before 7 December even if you started today, I think. MPEG-4 Visual works in an MP4 container and IIRC that worked well on Apple devices. (the primary problematic platform for this) And even software decoding is cheap as chips as it's 20+ years old. I thought there was an existing ticket for that, though I can't find it. (pretty sure brion posted some test files in it, maybe he remembers)
Question though: what is this ticket about? What is a "primary Wikimedia video format"? What is served to end users or something else? For the source/original file storage format AV1 could make sense, especially considering the file size limits imposed by swift. But this depends on the individual user (or app/Video2Commons/etc) AFAIK.
Question though: what is this ticket about?
AV1 does work with MediaWiki - with one exception: Thumbs are not generated for AV1!
Users usually never get the uploaded video, but a transcoded version. In the past there were transcodes to ogg and vp8. Now there are transcodes to VP9stream and VP9webm. AV1 transcodes could be added or replace one of the VP9 transcodes and a AV1 transcode could be made the preferred choice to offer to users.
The primary video format is the one we'll serve to readers by default, unless we detect that they can't manage it and so switch down to a different format. Currently this is WebM in VP9/Opus (formerly VP8/Vorbis), with no fallback except for bandwidth (or in some cases the original).
You can see this e.g. at https://commons.wikimedia.org/wiki/File:San_Francisco_earthquake_and_fire,_April_18,_1906.webm in the table of transcoded files from the original. Note that there are a bunch of the old confusingly-named "WebM" VP8 files; dropping these is T309823.
This change would add AV1 as the primary transcoding target, leaving VP9 available for readers whose machines can't cope with AV1.
For the source/original file storage format AV1 could make sense, especially considering the file size limits imposed by swift. But this depends on the individual user (or app/Video2Commons/etc) AFAIK.
Yes, this is nothing to do with editor/uploader activity. Obviously users should continue using Ogg Theora or whatever works for them! We shouldn't ever get in the way of users without very good reason.
Yes, this is nothing to do with editor/uploader activity.
No, it does. I do upload video files. for example: File:CSD_Hamburg_2022_001_part_1_of_26.webm
With AV1 encoding of the uploaded file the upload would be twice as fast.
Yes, but adding new support for AV1 uploads is T209440. This is about changing what we transcode to and push to readers, not what is accepted as an upload format.
not what is accepted as an upload format.
Actually AV1 is accepted with uploads. Only there are no thumbs for AV1. Not for the description page, not for the uploaded files page, not in wikipedia pages. The thumb creation fails with AV1.