Related to T350369: Errors in mw-client-NEW-errors TypeError: Cannot read properties of undefined (reading 'Keys')?

This issue is independent to T350369, because rEPHN365e80b9f580: Use static integers instead of OO.ui.Keys (https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Phonos/+/973387/) is now deployed, but issue still exist.

// Not loaded yet, but has a src URL so use that.
if (!this.audio && this.getHref()) {
    this.pushPending();
    this.audio = this.getAudioEl(this.getHref());
    // Play once after loading.
    this.audio.addEventListener('canplaythrough', () => {
        this.popPending();

        // Record the duration once to reduce access to Audio API.
        this.duration = this.audio.duration;
        if (this.usesAnimation) {
            this.$button.css('animation-duration', (this.duration / this.audio.playbackRate) + 's');
        }

        this.audio.play();
        // Track completion time
        const finishedAt = mw.now() - startedAt;
        this.track('timing.MediaWiki.extension.Phonos.IPA.can_play_through', finishedAt);
    }, {
        once: true
    });
}

This relies on canplaythrough, but media elements are not guaranteed to be loaded (depends on preload attribute, which has user-agent dependent defaults). Only after loading has begun, can the state become 'canplaythrough'. As there is no load, there is no canplaythrough triggered and the play will thus not be executed either.

There should be a audio.load(); after the src is assigned to the new Audio element. Technically that might have to be gated on the readyState, but i suspect thats not really needed, as the load will be done immediately after setting the src

Change 975095 had a related patch set uploaded (by TheDJ; author: TheDJ):

[mediawiki/extensions/Phonos@master] Load the audio src so that canplaythrough is triggered

https://gerrit.wikimedia.org/r/975095

Test wiki created on Patch demo by TheDJ using patch(es) linked to this task:
https://patchdemo.wmflabs.org/wikis/274381f11d/wiki/Hymne der Sowjetunion

The behavior on https://patchdemo.wmflabs.org/wikis/274381f11d/wiki/Hymne_der_Sowjetunion with a iPhone Safari is now slightly different:

• On a click on the button a loading animation is shown only for a short time, but then still nothing happens.
• A further click on the button starts playing the music.

I expect that the music should start already on the first click like on the desktop browsers.

The only other thing I can imagine that could still cause that is that the play action, isn't called quickly enough (because the device takes too long to download enough data). In that case the play is too far away in time from the user clicking play, which makes it not clear the permissions to play (playing audio requires a direct user interaction, within a certain timeframe).

1. is this a new phone, or an older one ?
2. what is your connection speed like, are pages loading intantly, or can you see them loading ?
3. are you using the mobile version of the page, or the desktop version of the page on your device ?

I tested with a iPhone SE3 with iOS 16.7.2, 30 MBit/s downstream and with mobile version and desktop version of the page. I can see the starting and stopping of loading animation, which takes a few 100 ms I guess.

Then the only difference I see is that I'm on iOS 17... shoot.

@brion do you have an older device laying around ?

The behavior on https://patchdemo.wmflabs.org/wikis/274381f11d/wiki/Hymne_der_Sowjetunion with a iPhone Safari is now slightly different:

• On a click on the button a loading animation is shown only for a short time, but then still nothing happens.
• A further click on the button starts playing the music.

This is exactly what happens with Brave when "Autoplay" is set to "Blocked" (but not with Edge or Firefox; Chrome has removed the equivalent setting). It might be because, after T345414, the playback is triggered asynchronously so it's not recognized as user-initiated.

Might be relevant: https://developer.chrome.com/blog/autoplay/#web-audio

Test wiki created on Patch demo by TheDJ using patch(es) linked to this task:
https://patchdemo.wmflabs.org/wikis/626676f327/wiki/Hymne der Sowjetunion

Test wiki on Patch demo by TheDJ using patch(es) linked to this task was deleted:

https://patchdemo.wmflabs.org/wikis/626676f327/w/

Test wiki created on Patch demo by TheDJ using patch(es) linked to this task:
https://patchdemo.wmflabs.org/wikis/4d880400da/wiki/Hymne der Sowjetunion

@Nardog can you try this ? https://patchdemo.wmflabs.org/wikis/4d880400da/wiki/Hymne_der_Sowjetunion

I'm forcing the clickhandler to play another (silent) audio element before the actual module is loaded. I think that might fix the iOS issue, but it doesn't fix the brave issue.

Brave seems…. weird. Even if we have already played audio, and you try to play another audiofragment, it seems to still block. So it’s not blocking auto play, it’s blocking ANY asynchronous play, for the first time of each audio element in the page. That’s not how that ever was supposed to work.

@Nardog can you try this ? https://patchdemo.wmflabs.org/wikis/4d880400da/wiki/Hymne_der_Sowjetunion

Same as before - requires two clicks. The point I was trying to make wasn't that it should be made to work in Brave as expected (given its pretty aggressive blocking and filtering, which you point out) but that the issue in question may hinge on some user configuration in Safari.

Change 975095 merged by jenkins-bot:

[mediawiki/extensions/Phonos@master] Load the audio src so that canplaythrough is triggered

https://gerrit.wikimedia.org/r/975095

Wait, I thought that patchset was just for demonstrative purposes...

Change 981393 had a related patch set uploaded (by TheDJ; author: TheDJ):

[mediawiki/extensions/Phonos@master] Load the audio src so that canplaythrough is triggered

https://gerrit.wikimedia.org/r/981393

Change 981393 merged by jenkins-bot:

[mediawiki/extensions/Phonos@master] Load the audio src so that canplaythrough is triggered

https://gerrit.wikimedia.org/r/981393

Current state: should work on the latest iOS version
Remaining problem: the click event is handled asynchronously, causing the 'direct userinteraction' / isTrusted context to get lost.

The problem is that the <audio> element is loaded fully asynchronous and the play is as well. This doesn't work well with the autoplay protection of platforms. Especially on slightly older browsers, which don't have a trusted timeframe yet, this is problematic.

Solution should be to create an audio element in the dom, BEFORE loading the player. Make sure that is directly played by a user interaction. This 'should' bust the autoplay. It does seem however that iOS applies this rule to ALL content, regardless of it being the first or the seconds played audio element. For this, we could create an audio element in the initial phase, then pass that to the 'player' and just change the source. In theory.. hard to test and verify because we need an older phone to test with, and there's also straight up bugs in Safari's .play() promise so....

If all else approaches fail: If .play fails, attach an error handler, and show a popup, asking the user to click again.

Should this ticket move to the Language team?

Should this ticket move to the Language team?

Yes, I think so.

Ack. We're handling Phonos tasks in the Phonos board until we pick them up for sprints.

Test wiki on Patch demo by TheDJ using patch(es) linked to this task was deleted:

https://patchdemo.wmflabs.org/wikis/274381f11d/w/

Test wiki on Patch demo by TheDJ using patch(es) linked to this task was deleted:

https://patchdemo.wmflabs.org/wikis/4d880400da/w/