Page MenuHomePhabricator

Score audio cannot be played if the page has other embedded media
Closed, ResolvedPublicBUG REPORT

Description

Steps to replicate the issue (include links if applicable):

What happens?:
Hovering over the media player shows https://en.wikipedia.org/wiki/File:Undefined as the link target; trying to play it produces an indeterminate progress bar (which somehow ends up behind the main article content, at least on Vector 2022), without actually playing anything.

image.png (1,135×454 px, 135 KB)

image.png (3,840×579 px, 139 KB)

What should have happened instead?:
The audio should be normally playable.

Other information (browser name/version, screenshots, etc.):
Minimum reproducible wikitext:

<score vorbis="1">
{a}
</score>
[[File:Example.ogg]]

Event Timeline

Minimum reproducible wikitext:

<score vorbis="1">
{a}
</score>
[[File:Example.ogg]]

I can reproduce anywhere with that code.

The issue has also been reported on itwiki.

Somehow, the same code works here but here doesn't

That task only reports inconsistent behaviour. The indeterminate progress bar issue is occurring in that example.

I was just explaining why it works in your first example and it doesn't in the latter. The presence of other embedded media on the same page triggers the use of the VideoJS player for Score-generated audio.

Nardog renamed this task from Bad Score output (File:Undefined) on enwiki: Ragtime progression to Score audio cannot be played.Apr 29 2024, 10:38 PM
Nardog edited projects, added: VideoJS player; removed: TimedMediaHandler.

If you add ?useskin=monobook&debug=1 you see the blue progress bar that is obscured in latest default Vector. When I click the broken score player's play button I see jQuery.Deferred exception: title is undefined in Firefox browser console, coming from doPropagation of a requestIdleCallback handler that some module is ready.

Dubious source code analysis follows...

The mw-ext-score <div> contains an <audio> tag with

<source src="//upload.wikimedia.org/score/t/s/tsi3u4jmg0p4ocq22jukqyia6o9es1o/tsi3u4jm.mp3" type="audio/mpeg">

TMH runs and wraps this with its

<span class="mw-tmh-player audio mw-file-element"><audio playsinline="" preload="metadata" class="video-js">

When you click play, TMH puts up a <div class="mw-tmh-player-interstitial"> and calls infuse() in in https://test.wikipedia.org/w/load.php?debug=1&lang=en&modules=ext.tmh.player.inline%7Cext.tmh.video-js&skin=monobook&version=ztntf which "Takes the HTMLMediaElement of the InlinePlayer and infuses it with JS (videoJS) to enrich the element." At line 192 of this

		const resource = this.videoplayer.getAttribute( 'resource' );
		const resourceTitle = resource ?
			decodeURIComponent( resource.slice( resource.lastIndexOf( '/' ) + 1 ) ) :
			this.$videoplayer.data( 'mwtitle' );
		this.playerConfig.mwTitle = mw.Title.newFromText(
			resourceTitle, NS_FILE
		);

There's no resource, so it tries to use this.$videoplayer (Note $videoplayer not videoplayer or videojsPlayer); but I think the $videoplayer isn't set up yet, so ??? weird deferred promise stuff happens. Anyway, resourceTitle ends up undefined (note, not null), and then mwTitle.newFromText() is presumably unhappy. If I break on exceptions in Firefox's debugger, there's an exception in

parse = function ( title, defaultNamespace ) {

called from

Title.newFromText = function ( title, namespace ) {

because parse() is fiddling with an undefined value.

Anyway, eventually the jQuery.Deferred.exceptionHook runs and it prints "jQuery.Deferred exception: title is undefined". Interestingly Title.newFromText() has the comment "Constructor for Title objects with a null return instead of an exception for invalid titles", but it does trigger an exception if the title isn't a string.

My crappy analysis suggests two possible fixes.

  1. Make Title.newFromText() even more robust; it or parse() can check if (typeof (title) !== 'string' and bail. However, later on this causes grief in InfoButtonPlugin which naively tries to access title.getMainText() when title is null, so that would have to be tweaked as well. I think this makes a button that links to the mediawiki File: object for the audio file, but there isn't an info button in an inline videoplayer, so it seems wasted work in this case.
  2. Give this.videoplayer a resource attribute, e.g. at line 119 in infuse() where it gives it a preload attribute. If before line 192 I push a resource attribute into this.videoplayer in the Firefox debugger by running this.videoplayer.setAttribute('resource', 'dummy'); there's no exception, the browser's audio player is replaced by this videoplayer, and the music plays. Maybe setting the resource attribute to the <source src> attribute would be more correct.

Hope this helps; I have little idea WTH is going on.

Nardog renamed this task from Score audio cannot be played to Score audio cannot be played if the page has other embedded media.May 18 2024, 12:55 AM

Another possible fix:

  1. ext.tmh.player.element.js's load() bails if a parent of the <audio> tag has class=mw-tmh-player, since "This player has already been transformed". So Score's generateHTML() could add this class to the audio tag and it wouldn't be turned into a TMH player. I tried this in the Firefox debugger and it fixes the bug. Obviously it would be better if TimedMediaHandler and Score cooperated on some explicit attribute that indicates "don't transform this audio/video tag into a TMH player."

TMH should set its own class attributes on the server side and avoid mangling unrelated content. Per my commit message comments on cf8a4f2133cb, TMH is too broken to use with Score. Requiring every other extension to set a fake TMH attribute to trick TMH into not breaking it doesn't seem like a good solution.

Thanks Tim!

  1. FWIW this change to TMH makes it ignore the audio tag that Score generates.
diff --git a/resources/ext.tmh.player.js b/resources/ext.tmh.player.js
index adca787f..0ad9e8a6 100644
--- a/resources/ext.tmh.player.js
+++ b/resources/ext.tmh.player.js
@@ -26,7 +26,7 @@ $.fn.loadVideoPlayer = loadVideoPlayer;
  * @private
  */
 function loadVideoPlayers( $content ) {
-       $content.find( 'video, audio' ).loadVideoPlayer();
+       $content.find( 'video, audio.mw-file-element' ).loadVideoPlayer();
 }

Change #1040745 had a related patch set uploaded (by TheDJ; author: TheDJ):

[mediawiki/extensions/TimedMediaHandler@master] Handle missing resource or mwtitle

https://gerrit.wikimedia.org/r/1040745

Change #1040745 merged by jenkins-bot:

[mediawiki/extensions/TimedMediaHandler@master] Handle missing resource or mwtitle

https://gerrit.wikimedia.org/r/1040745

Change #1039696 had a related patch set uploaded (by Fomafix; author: Fomafix):

[mediawiki/extensions/TimedMediaHandler@master] Add attribute data-mw-tmh="" to the video/audio elements

https://gerrit.wikimedia.org/r/1039696

Change #1040725 had a related patch set uploaded (by Fomafix; author: Fomafix):

[mediawiki/extensions/TimedMediaHandler@master] Use '[data-mw-tmh]' as selector instead of 'video, audio'

https://gerrit.wikimedia.org/r/1040725

TheDJ claimed this task.

Fixed on https://test.wikipedia.org/wiki/Score_and_media and will deploy to remaining servers over the week. Moving remaining patches over to the other ticket, which deals more with the issue of what <audio> and <video> element belong to the which backend extension/frontend script.

Change #1091871 had a related patch set uploaded (by TheDJ; author: TheDJ):

[mediawiki/extensions/TimedMediaHandler@master] Remove old mw-score styling that is no longer needed

https://gerrit.wikimedia.org/r/1091871

Change #1091871 merged by jenkins-bot:

[mediawiki/extensions/TimedMediaHandler@master] Remove old mw-score styling that is no longer needed

https://gerrit.wikimedia.org/r/1091871