List of steps to reproduce (step by step, including full links if applicable):
What happens?:
What should have happened instead?:
The behavior should be consistent.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | Feature | None | T301156 Request: Smaller play bar for scores | ||
| Resolved | BUG REPORT | Fomafix | T295514 Score audio uses different players depending on whether the page has other embedded media | ||
| Resolved | BUG REPORT | TheDJ | T363630 Score audio cannot be played if the page has other embedded media |
This task is about how TimedMediaHandler's player is sometimes used for Score and at other times not (depending on the content of the rest of the page), whether VideoJS is enabled or not. That VideoJS doesn't work with Score is a completely separate issue.
I think the problematic code is in ext.tmh.player.js:
mw.hook( 'wikipage.content' ).add( function ( $content ) { $content.find( 'video, audio' ).loadVideoPlayer(); } );
The main questions I have are:
cc @TheDJ and @brion as TMH experts.
Change 740942 had a related patch set uploaded (by TheDJ; author: TheDJ):
[mediawiki/extensions/TimedMediaHandler@master] Make dialog not crash on missing title
Either VideoJS/Kaltura always works for score (preferably without delay)
The JS players always introduce delay, because they have to be loaded asynchronously.
Given rESCRcf8a4f2133cb: Drop TimedMediaHandler support, the latter seems to be the intended behavior, but VideoJS/Kaltura replaces the HTML5 player whenever the page has other media embedded using [[File:...]].
Just because you don't insert the player modules, doesn't mean of course that other elements don't insert the players. There is also no specific opt out for TMH right now. So that commit was bound to break things.
TMH just enriches HTML5 audio, video elements (all of them). Score loses consistency, because there isn't an explicit op-out from this behavior.
Score also loses visual consistency, as HTML5 audio players look different on all platforms. HTML 5 audio players were never rly intended to be for consumer use, so they are not standardized in terms of looks/functionality/dimensions.
One of the big problems of TMH is that it is lazy loaded by click, instead of lazy loaded upon load. This was done as a performance enhancement for pages with lots of video players on them. This new lazy loading has the unfortunate disadvantage, that you begin with a working player than can start the playback (if you are quick), which you then (visually) hide and replace with a button, which you then have to click and wait to load a player again in which you can play. (Maybe that should become a config option or something, because i think it's not really an option anyone but WMF actually desires).
- If so, how should we communicate that, add some no-tmh-please CSS class to the <audio> element?
That's an option mw-tmh-optout. Or have TMH opt-in and only process it's own videos. My personal original intent was that that should not be needed, and that actually the transcode and transform pipeline and the player should be completely independent of each other and possibly even be separate extensions. Then you could just switch between playback extensions when you wanted).
But this is something that someone should think through, and decide upon and also consider the new html layout that parsoid introduced etc...
The JS players always introduce delay, because they have to be loaded asynchronously.
But not like five seconds, which it takes for the Kaltura player to show up for the score audio.
Yes the kaltura player is extremely slow, part of why I've been trying to replace it for over 6 years now.
But it does show up pretty much immediately for uploaded media ([[File:...]]). (Not that it matters if you're going to disable it for Score, but.)
A quick check highlights that because the score tag doesn't feature the 'duration' attribute, Kaltura sets a 'waitformetadata' flag. This causes it to delay initialisation till after the 'loadedmetadata' event. This event either never triggers, or was already triggered before the event was installed. A fallback is in place at 5 secs to force the player to load.
Further checks. The audio metadata is already loaded before the event handler is installed. The waitformetacheck function should probably check the element.readyState to be higher than 0 instead (all this was written before HTML5 was actually available, so it's all kludges).
Change 740942 merged by jenkins-bot:
[mediawiki/extensions/TimedMediaHandler@master] Make dialog not crash on missing title
Change 741702 had a related patch set uploaded (by TheDJ; author: TheDJ):
[mediawiki/extensions/TimedMediaHandler@master] Check if the native player already has the metadata loaded
with these two patches at least score will work again. It doesn't solve the inconsistency of the players that are in use.
An opt-out class just like MediaViewer's noviewer and CirrusSearch's navigation-not-searchable would be great.
It can be done just by adding something like .not( '.no-embed-player *' ) to this line.
Change 741702 merged by jenkins-bot:
[mediawiki/extensions/TimedMediaHandler@master] Check if the native player already has the metadata loaded
The 5sec. delay and the videojs crash are both fixed now. The inconsistency however is still there.
Well, whether it uses Kultura or VideoJS simply reflects site/user settings, so that need not be consistent.
I thought this has long been fixed. In the score+media case Score’s audio output simply disappears, making it impossible to play it. Since Video.js is already default on some wikis (which shouldn’t have happened without this bug being fixed, by the way), this should be fixed ASAP—either fix it really, or roll back everything to Kaltura, which is also inconsistent, but at least works (Kaltura provides a working UI for Score’s output). As a production example, https://fr.wikipedia.org/wiki/Chaconne_en_r%C3%A9_mineur_(Pachelbel) contains a score that should have MP3 output, but it disappears.
This is a separate issue (and should be filed separately). The player loading broke when we switched the audio mode of videojs to become an inline loaded element.
It’s technically covered by this task’s title—the player is different depending on whether the page has other embedded media, it’s a native one when there’s no other player, and an invisible one when there’s one. 🙂 However, I see that the underlying issue is different. Fortunately T306836 was fixed since then, so we’re back at the original issue of players appearing in both situations, but different ones. This means that now it’s functional, so it’s not ASAP anymore, but it should still be fixed, and my opinion on where to fix it didn’t change since last week:
I mentioned a possible hack workaround in T363630:
ext.tmh.player.element.js's load() bails if a parent of the <audio> tag has class=mw-tmh-player, since "This player has already been transformed". So Score's generateHTML() could add this class to the audio tag and it wouldn't be turned into a TMH player. I tried this in the Firefox debugger and it fixes the bug. Obviously it would be better if TimedMediaHandler and Score cooperated on some explicit attribute that indicates "don't transform this audio/video tag into a TMH player."
It seems what's missing here is a spec for desired behavior, in both the short-term and "ideal world." Should a score with audio use VideoJS or use the browser's <audio> tag implementation? Should it have an info button linking to the raw generated .mp3 file? If the score has lyrics, should they appear as closed captions (I think this would require the score being an external File:). Etc.
Change #1040725 had a related patch set uploaded (by TheDJ; author: Fomafix):
[mediawiki/extensions/TimedMediaHandler@master] Use '[data-mw-tmh]' as selector instead of 'video, audio'
Change #1039696 had a related patch set uploaded (by TheDJ; author: Fomafix):
[mediawiki/extensions/TimedMediaHandler@master] Add attribute data-mw-tmh="" to the video/audio elements
Change #1041732 had a related patch set uploaded (by Arlolra; author: Arlolra):
[mediawiki/services/parsoid@master] Add attribute data-mw-tmh="" to the video/audio elements
Change #1039696 merged by jenkins-bot:
[mediawiki/extensions/TimedMediaHandler@master] Add attribute data-mw-tmh="" to the video/audio elements
Change #1041732 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Add attribute data-mw-tmh="" to the video/audio elements
Change #1043324 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.20.0-a8
Change #1043324 merged by jenkins-bot:
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.20.0-a8
Both implementation are now outputting the attribute, just need to wait for parser cache to roll over.
Change #1040725 merged by jenkins-bot:
[mediawiki/extensions/TimedMediaHandler@master] Use '[data-mw-tmh]' instead of 'video, audio' as selector