Page MenuHomePhabricator

Some audio formats are not playable in Phonos on iOS/MacOS/Safari
Closed, ResolvedPublic5 Estimated Story Points

Description

Background

Phonos allows you to link to an audio file without providing any IPA, i.e. <phonos file="Example.ogg" />. This on the surface would seem to satisfy the Audio links that play on click wish from the 2022 survey (T229169). The issue however, for both that wish and Phonos, is that some formats such as OGG are not natively playable on MacOS/iOS/Safari or some combination thereof. On WMF wikis, TimedMediaHandler takes care of this for you. Phonos does not, so these users will get no audio.

Acceptance criteria

  • Use the TimedMediaHandler backend to fetch MP3 derivatives, if they exist
  • If no MP3 derivative exists, serve the original file format instead

QA notes

  • Find an existing file that does not have an MP3 extension. Using it in Phonos, it should still play an MP3, if that derivative exists. It usually does, but not always.
  • Examples of files with MP3 derivatives:
    • Example.ogg
    • Voiceover-mathml-example-1.wav
    • Wikidata items Q1 and Q2
  • Example of a non-MP3 file with no MP3 derivative:
    • Or-ଓଡ଼ିଆ 01.oga

You can check if MP3 derivatives are available by looking under the "Transcode status" section on the file page. I.e. at File:Or-ଓଡ଼ିଆ_01.oga you'll notice there is an MP3 derivative, but it is marked as "Not ready", hence why we don't get an MP3 in Phonos. Example.ogg on the other hand shows the MP3 derivative is available.

Using Special:MediaSearch on Commons and choosing a file format is an easy way to find more examples. The above examples may not apply to Beta.

Event Timeline

MusikAnimal renamed this task from Some file types are not playable in Phonos on iOS/MacOS/Safari to Some audio formats are not playable in Phonos on iOS/MacOS/Safari.Nov 3 2022, 7:48 PM
MusikAnimal added a subscriber: Nardog.

Yeah, it should definitely fetch derivative URLs and create an HTMLSourceElement for each.

Why do we need to include each? MP3 is supported everywhere, no? At least for the purposes of Phonos (which is why I ask here and not at T229169), the goal is to make it playable for everyone. If converting to MP3 suffices, that should cover us from a product standpoint. But we'd love to hear any reasons why other formats might be preferred.

Maybe it isn't imperative that all derivatives are provided, but my point is that the behavior should be more or less consistent with what [[File:...]] yields.

@MusikAnimal

Firefox 106 was able to play repeatedly, while Chrome and Edge 107 were only allowed to play once as seen in the screenshot below from the test link below. I will create a separate task for that in case this fix doesn't resolve it. Also I'm sure the Safari issue is a known one but added a screenshot too.

As for the mobiles, here are my findings:
Pixel 6 with Firefox, it was able to play all sounds but on Chrome it didn't.
Pixel 7 and Galaxy S22 it didn't play sounds on either Firefox or Chrome.
Iphone 14- same as Safari on Mac

Test link: https://en.wikipedia.beta.wmflabs.org/wiki/Phonos_without_any_IPA

'Chrome 107 and Edge 107'

T314834_Phonos_Chrome_Edge_OGG_FAIL.png (1×2 px, 151 KB)

Safari 15.6

T314834_Phonos_Safari_FAIL.png (961×1 px, 344 KB)

You need TimedMediaHandler to upload files anyway, so why can't <phonos file="..."> simply convert to a wrapper element with [[File:...]] in it (but Phonos taking over the playback instead of VideoJS)? That way Phonos doesn't have to have its own code for fetching derivatives.

You need TimedMediaHandler to upload files anyway, so why can't <phonos file="..."> simply convert to a wrapper element with [[File:...]] in it (but Phonos taking over the playback instead of VideoJS)? That way Phonos doesn't have to have its own code for fetching derivatives.

Something like that might work, but regardless I had a look into the TimedMediaHandler backend the other day and it seems there are public methods available to do the transcoding for us, as we hoped. I just haven't tested it yet. That might the more appropriate route, and should still mean we don't have to fetch or generate derivatives in Phonos. I'll have more info soon after I investigate further.

This comment was removed by Nardog.

Change 869835 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[mediawiki/extensions/Phonos@master] Use TimedMediaHandler to find MP3 derivatives for non-MP3 files

https://gerrit.wikimedia.org/r/869835

Change 874439 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[integration/config@master] zuul/parameter_functions.py: Make Phonos depend on TimedMediaHandler

https://gerrit.wikimedia.org/r/874439

Change 874439 merged by jenkins-bot:

[integration/config@master] zuul/parameter_functions.py: Make Phonos depend on TimedMediaHandler

https://gerrit.wikimedia.org/r/874439

Change 874964 had a related patch set uploaded (by Dmaza; author: Dmaza):

[integration/config@master] zuul/parameter_functions.py: Make phan work for Phonos

https://gerrit.wikimedia.org/r/874964

Change 874964 merged by jenkins-bot:

[integration/config@master] zuul/parameter_functions.py: Make phan work for Phonos

https://gerrit.wikimedia.org/r/874964

Change 869835 merged by jenkins-bot:

[mediawiki/extensions/Phonos@master] Use TimedMediaHandler to find MP3 derivatives for non-MP3 files

https://gerrit.wikimedia.org/r/869835

Pending feedback on playing a .oga file if they are getting the same results in Safari 16.0

Link tested: https://en.wikipedia.beta.wmflabs.org/wiki/Phonos_without_any_IPA
OS: Mac 13.0
Broswer: Safari 15.6 & 16.0
Environment: Beta

Safari 16.0- App: Downloads it to the Download folder first.

T322368_IPInfo_AudioFormats_Safari16_0.png (847×1 px, 140 KB)

Safari 15.6- BroswerStack: plays directly

T322368_IPInfo_AudioFormats_Safari15_6_BrowserStack.png (1×1 px, 388 KB)

@MusikAnimal Audio files tested are .mp3, .wav, .mid, .opus, .midi, .webm, .flac, .oga, and .ogg. All audio file formats with MP3 derivatives were playable on macOS (Safari & Chrome) and on an iPhone 14 (Safari & Chrome).

Test link: https://en.wikipedia.beta.wmflabs.org/wiki/Phonos_Files

OS: Mac 13
Browser: Safari 16.0 & Chrome 108
Environment: Beta

macOS: Safari -no MP3 derivative

T322368_Phonos_AudioFormats_MacSafari_NoMP3Derivative.png (1×2 px, 767 KB)

macOS: Chrome -no MP3 derivative

T322368_Phonos_AudioFormats_MacChrome_NoMP3Derivative.png (1×3 px, 479 KB)

iPhone 14: Safari -no MP3 derivative

T322368_Phonos_AudioFormats_IP14Safari_NoMP3Derivative.png (1×2 px, 656 KB)

iPhone 14: Chrome -no MP3 derivative

T322368_Phonos_AudioFormats_IP14Chrome_NoMP3Derivative.png (1×2 px, 808 KB)

MusikAnimal set the point value for this task to 5.Jan 17 2023, 5:59 PM