Page MenuHomePhabricator

Migrate primary Wikimedia video format from VP9 to AV1
Open, Stalled, Needs TriagePublic

Event Timeline

Jdforrester-WMF changed the task status from Open to Stalled.Nov 14 2018, 12:23 AM
Jdforrester-WMF created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 14 2018, 12:23 AM
brion added a subscriber: brion.Nov 14 2018, 12:44 AM

Note that this is dependent on suitable playback support being widespread. While on desktops, Chrome and Firefox are shipping AV1 now or in next versions, and Microsoft is previewing it for Edge, we don't yet know the situation for Safari. I've started adding AV1 decode to ogv.js, but it's slower than decoding VP9 so far so may require running lower resolutions (unless WebAssembly threading arrives soon on Safari).

For mobile, currently no phones have AV1 hardware support, though Android Chrome may get it in software. Eventually it's expected to be more widespread, at least as much as VP9 is today, but this is not something we can be too aggressive on.

It's hard to say when the right time is, but waiting for more widespread support is a good idea. Imagine we start transcoding now and when hardware implementations finally come around, nothing plays because their implementation is quirky.

Also, we should compare VP9 and AV1 ourselves. I trust few third party evalutations and those that I do trust are absolutely not Google, some university and Facebook.

And who am I to say that? Well, I've encoded movies with DivX 3.11. 'Nuff said.

djewell added a subscriber: djewell.EditedJan 10 2020, 2:19 AM

As a follow-up to @AlexisJazz and @brion's comments above, AV1 hardware decoding support is very limited right now - see en-wiki:AV1#Hardware for some notes. Even VP9 and HEVC/x265 hardware decoding is relatively new ... NVidia has a page on their dev site that talks about hardware support. Onboard Intel graphics chipsets aren't much better - yes, the newest models can typically handle it, but even 3-4 years old (or the lower-end low-power stuff) might not.

Speaking from experience trying to watch videos on YouTube served up (by default) in VP9 on a computer that doesn't have H/W decoding for VP9 (but does for avc1/h.264), it's pretty much impossible to comfortably watch a video higher than 480p resolution (not to mention the CPU pegging at 100% and the associated battery drain). Switching to h.264/mp4 and the H/W decoder kicks in -- even 4k -- no problems.

Is it possible to detect H/W decoding capabilities in the browser? (I'm thinking out-loud -- I've never checked and just now thought about it.) It would be interesting to try and figure out the current capability set that's out there... and use that data as a guide on when to switch? (I'm guessing the answer is no, because it would seem like YouTube would be able to detect that and automatically serve up whichever format is hardware-decoding supported on the user's machine - it doesn't appear to do that... but that's assuming that they would do that. :) )

Quick edit: Yes, there's an API for that ;) - not supported on all browsers, but modern ones, yep! https://developer.mozilla.org/en-US/docs/Web/API/Media_Capabilities_API

brion added a comment.Jan 13 2020, 5:35 PM

Native CPU-run VP9 seems pretty ok through 720p on my lower-spec machines, but YMMV. :) AV1 is much more CPU-intensive and I definitely don't recommend switching to it wholesale yet, especially at high resolutions.

Use of Media Capabilities to pick a preferred format is a definite possibility, for supporting browsers. (It's still new so not everything has it reliably, IIRC, but it should cover current Chrome and Firefox and that's most of the world.)

(Note that the reason we're not on h.264 is the patent licensing requirement; until that gets resolved -- either by waiting until 2028 for the patents to expire or overturning the 2014 Commons RFC and reinterpreting or changing the foundation's open file formats policy -- we are stuck working around it with other formats.)

Alternate possibilities for generating files also include generating AV1 only for low resolutions (where the bitrate savings make the most difference and the CPU cost makes the least), which I believe was YouTube's first step on deploying AV1. Anyway no hurry, we don't have to be an early adopter on this when we have such limited resources to apply for now...