Huh, I already did this in 2018: c6b601c4b

... a couple months after these particular transcodes were made. ;)

Ok, I'll re-run just this file for now :D

Ugh, even a newer transcode isn't running. Maybe it doesn't like the 5.1 audio?

Yep, squashing from 5.1 channels to 2 channels resolves it. \o/

Ok, lemme fire up a patch.

Change 773596 had a related patch set uploaded (by Brion VIBBER; author: Brion VIBBER):

[mediawiki/extensions/TimedMediaHandler@master] Force 2-channel audio squashing on transcodes for Safari compat

https://gerrit.wikimedia.org/r/773596

Upstream ticket: https://bugs.webkit.org/show_bug.cgi?id=229325

Its kinda sad that we would lose multichannel audio support for Firefox/Chrome platforms that DO properly support media files though... Then again, it doesn't seem as if it is in use a whole lot, so maybe for now it's acceptable ?

Filed https://bugs.webkit.org/show_bug.cgi?id=238357 for the VP9 profile 1 and 3 support (non 4:2:0 chroma types). Not holding my breath ;)

What if... We intercept the MEDIA_ERR_SRC_NOT_SUPPORTED error, then throw that source out of our src list and select the next source candidate ? Then we would get a VP8, vorbis source, which would work.

Hrm, I hate having to replicate base browser behavior as workarounds lol. ;)

We could also just use Vorbis audio with the VP9 output in a pinch, if one works and the other doesn't. I'll take another poke just in case I can make it work...

i seriously doubt that Safari has multichannel vorbis ;)

Ok, confirmed Safari likes 1 and 2-channel but not 5 or 6 (5.1) channel Opus.
It plays back a Vorbis 5.1 channel file but I don't know if the channel assignments or correct or anything (I have no suitable hardware).

I'm inclined to stick with 2-channel forcing for now and worry about proper multichannel upgrades later...

Change 776001 had a related patch set uploaded (by TheDJ; author: TheDJ):

[mediawiki/extensions/TimedMediaHandler@master] Attempt at doing source selection fallback

https://gerrit.wikimedia.org/r/776001

Test wiki created on Patch demo by TheDJ using patch(es) linked to this task:
https://patchdemo.wmflabs.org/wikis/b0a0e446ec/w/

In T304624#7811823, @brion wrote:

Hrm, I hate having to replicate base browser behavior as workarounds lol. ;)

I experimented anyway ;)

I still have to test a bit more, but one of the things with falling back to VP8 codecs that I noticed: The native VP8 decoder seems to be... not that good. It often freezes (on I-frames I suspect), and takes a long time to recover (and that is on my 2021 MacBook Pro). I suspect this is because it is a software decoder, and not a really good one. :(

BTW. i was listing to this clip on my airpods, and noticed that ogvjs doesn't handle multichannel opus correctly either (i guess not totally surprising, we probably don't do any channel mapping, as that is a 'post codec' thing generally).

If I open the VP9 clip on my old mac with videojs vs with VLC media player, I can clearly notice that the left channel i hear with ogvjs is probably the rear left channel. Dialog is very much muted but does have music and surround sounds. Right channel does have dialog, but i'm not entirely sure if im listening to left (most likely probably), center or right.

https://upload.wikimedia.org/wikipedia/commons/transcoded/a/a2/Elephants_Dream_%282006%29.webm/Elephants_Dream_%282006%29.webm.480p.vp9.webm

In T304624#7839299, @TheDJ wrote:

BTW. i was listing to this clip on my airpods, and noticed that ogvjs doesn't handle multichannel opus correctly either (i guess not totally surprising, we probably don't do any channel mapping, as that is a 'post codec' thing generally).

Oh yeah, two likely problems:

• it's forcing to stereo for back-compat reasons (this can likely be lifted in the all-Web-Audio age)
• the channel mapping is almost certainly wrong :D

Original upstream issue on ogv.js is here: https://github.com/brion/ogv.js/issues/30 probably needs a double-check. It's also possible that Vorbis and Opus have different standard mappings, I haven't really checked. Ugh. Anyway follow-up there.

https://patchdemo.wmflabs.org/wikis/0396353644/wiki/File:Elephants_Dream_(2006).webm

Still a bit on the fence on where we should go with this. Have stereo audio for everything, or source fallback.... I think the amount of non-stereo sources is extremely low, so maybe it doesn't really matter if we lose that, at the same time. It just seems such a shame.

Change 776001 merged by jenkins-bot:

[mediawiki/extensions/TimedMediaHandler@master] Do source selection fallback

https://gerrit.wikimedia.org/r/776001

Test wiki on Patch demo by TheDJ using patch(es) linked to this task was deleted:

https://patchdemo.wmflabs.org/wikis/b0a0e446ec/w/

Change 773596 abandoned by Brion VIBBER:

[mediawiki/extensions/TimedMediaHandler@master] Force 2-channel audio squashing on transcodes for Safari compat

Reason:

Abandoning in favor of using Vorbis, where the >2-channel decoder works in Safari

https://gerrit.wikimedia.org/r/773596

For tracking: Safari 17.0 on macOS Sonoma adds support for one or two channel Opus audio in WebM and MPEG-4 containers
https://webkit.org/blog/14445/webkit-features-in-safari-17-0/

Given the "fun" of difficult-to-detect channel limitations and inconsistency in playback spport, looks like we'll be generating stereo output for now. :) But I've left room open on the HLS streaming tracks to add multichannel tracks alongside the stereo tracks, which can be marked separately with their channel count (assuming iOS actually pays attention to that for compatibility checking....)

Worth working on later, and this leaves us open for adding them when supported...

The WebMs are used in desktop Safari still for now, though, and may still encounter problems until we switch the default playback over to use HLS-over-MSE with the explicitly downmixed HLS tracks.

It probably wouldn't hurt to just go ahead and squoosh channels to 2 on the webms too, with the adaptive streaming tracks as the future escape hatch for full-quality output on compatible systems. Sigh. I'll prep a quick update patch.

Multichannel opus audio was allegedly fixed upstream in Safari this week.