In T68722#9356430, @TheDJ wrote:

BTW. I realised that HLS doesn't support media temporal fragments, so likely start and end params won't work on iOS. There is probably no easy way around that, we'll likely have to come up with something custom.

m3u8 has #EXT-X-START:TIME-OFFSET=-15, but as start/end are transclusion params, we can't easily embed them into the manifest without using dynamic manifests (currently we have static manifests)

Started bulk re-encoding with the remuxing being used transparently whenever possible. I expect full coverage at at least base resolution by end of calendar year. :D

Closing.

[In general I know we've got some order of operations problems with the transcode table and other db bits, the job queue, and the file storage -- three distinct storage systems which need to be either in sync or in the proper order to handle weird sync issues. I'm going to collect up a few of these related issues today and try to work them out in the next couple days. If we're lucky I'll have something up for code review before the Thanksgiving holiday Thursday! If not I'll keep poking at it later.]

I'll add this to my queue. Currently I'm chasing down bulk encodings; chasing down ongoing errors will be next and I expect several synchronization-related problems with cached data chunks. :)

Script completed. If no surprises during the next couple days of rebuilds, all should be well to resolve. :D

Script is up to the "An"s, I expect it to complete within ~24-48 hours.

Running a one-off rebuild of streaming manifests separately from the active transcodes. This should fix any that were damaged that haven't had a chance to be fixed by other runs yet:

It'll require actually updating one of the other transcodes on an affected file to rebuild the m3u8 to work again... I've got several batches running, but I might be able to force a simple regeneration too.

Fix deployed, these'll get cleaned up as transcodes run.

I think we can just take this bit out safely, it wasn't reliable and the requeue scripts have to handle the case where a row just got left anyway. :D Patch coming...

Oof, yeah that explains it. I'll add it to my pile of fixes for this week. :D

Nothing should be writing to the database from TimedMediaHandler in page views, as far as I know. I think there used to be something years ago but if it's still there it needs to be removed. Is this still a problem that anyone is aware of?

done

this finished :D

This still happens, and it's mildly annoying. ;) We may need to force proper chronology on the database connections used to fetch file data? Anyway, back on the slate for cleanup work :D

I changed my mind and, in recent updates, instead added extra bandwidth for the higher-frame-rate videos so they still compress nicely.

For .mov container input, which is reasonably common on certain generations of cameras, all we should need now is consistent support for ISO BMFF-family input files with proper codec validation (right now we kinda half-ass it). Depending on source files, some may have AAC audio as well (cf T166025).

Hm, maybe I misremember. I'll check this back next week, it still isn't playing in Safari and I'm not sure if it's a cache thing or a breakage thing and I'll deal with it Monday.

I've implemented the squooshing to 2-channel output in recent updates since We Can't Have Nice Things. Re-running transcodes on this file. See T351025 for the more permanent long-term solution of adding surround tracks probably using AAC

Current state of this:

When time permits I'm planning to do a big refactor on the MPEG and MP3 parsers for getid3 and submit that upstream; there's a lot of things that are legit in bitstreams but it doesn't handle properly.

Background run for backfill is going; this may take quite some time to complete, but I think should have an initial low-res data set within a few days for everything. I'll continue to monitor it while tweaking the app for a cleaner display & interaction, but basic playback should "just start working" in the iOS app over time :D

Short version would be to just swap OOUI dialogs for vue+Codex dialogs? Probably worth doing to future-proof.

*nod*; AAC could come in as a standalone .m4a or with MPEG-4 Visual video in .3gp files for instance; we'd need to get legal clearance on both but it should be straightforward to start working towards that (eg, making sure we can selectively check for codecs in an ISO BMFF/MP4/QuickTime-family media file and start allow-listing the ones we don't have a worry about)

It would be pretty trivial to enable H.264 and AAC output, we have all the necessary machinery ready to go for traditonal flat .mp4 files and can very easily add definitions for HLS. AAC may be good to go as far as patent expiration but there's not a big rush to enable it unless H.264 comes along for the ride, and that's still got some time yet.

Yep, that does it. :D Good catch!

Note that testing with the new iOS-compatible HLS track output is very promising so far! This is enabled on local media files on test.wikipedia.org only so far; playback is functional and supports native fullscreen and picture-in-picture. I'm taking this bug assignment to do some further work on the iOS app to tweak the styles to get larger videos sized nicely etc. :D

This should be fixed since the deployment circa 12 hours ago. Please confirm and resolve when you have a chance, thanks!

This should be fixed since the deployment circa 12 hours ago. Please confirm and resolve when you have a chance, thanks!

This should be fixed since the deployment circa 12 hours ago. Please confirm and resolve when you have a chance, thanks!

Does the legal team have a specific thing they're interested in investigating? (eg, are you claiming that a specific problem file was reported and could not be deleted, and that nobody forwarded the issue on to the ops team to expedite deployment of the fix? Or are you just thinking out loud?)

No practical use for 8k other than VR video, which we don't support, and even if we did handling the conversion would be extremely expensive. Declining.

Definitely regressions in the last few months since I last tested it, I'm adding this to my short-list for this week.

I could make these non-fatal errors I guess, just log and move on. :(

Oh I see it's saving empty playlists anyway heh. Well that's fine, it should work and be harmless...

Note there shouldn't be any streaming transcodes made for .wav files at this time, so something's amiss as well.

Is it simply unsafe to use locks in the database to ensure consistency between DB and filesystem?

So is this believed to be fixed by the recent fix? Or did the fix already go out and didn't fix this?

Not yet. wikimedia/getid3 isn't published yet, and production is still pinned to james-heinrich/getid3. Can you get it published so I can update vendor?

I'm running the cleanup batch script now on mwmaint2002

So if we add server-side rendering, we need a clean way to sandbox it, that works both in standalone installs and via Thumbor, and can be relatively sensibly managed and packaged, and made disableable with a switch-flip. I'm also hoping to work on some iframe sandboxing stuff for media embedding, which will later be usable in stuff like media renderers (which we can port the existing video & 3d interfaces into when it's ready)... I think we've got some tasks open for that related to Graphs, I'll track that down later :D

[Note: in general pulling in binary file parsers/renderers is a big extra security risk for server-side execution. Pulling in a large JavaScript rendering library is also a security risk for client-side safety, as it could expose an XSS injection vector in the library that can be used against the entire page. So there's a relatively high bar, and we've seen problems in production for real with Graphs etc.]

Note that improving media file format support is still not something that's explicitly receiving resourcing at Wikimedia Foundation, so it's good to have a low cost in terms of implementation work (note that this includes thumbnailing via thumbor, an entirely separate package), low risk in terms of tooling safety from libraries and executables used in rendering on the server or client side, an easy way to embed the resulting view into mobile apps without too much platform-specific coding, and an easy safe way to downgrade support if security problems are found in the future.

It probably wouldn't hurt to just go ahead and squoosh channels to 2 on the webms too, with the adaptive streaming tracks as the future escape hatch for full-quality output on compatible systems. Sigh. I'll prep a quick update patch.

Given the "fun" of difficult-to-detect channel limitations and inconsistency in playback spport, looks like we'll be generating stereo output for now. :) But I've left room open on the HLS streaming tracks to add multichannel tracks alongside the stereo tracks, which can be marked separately with their channel count (assuming iOS actually pays attention to that for compatibility checking....)

This turned out to be a problem with my test configuration, I was having trouble with positional parameters being passed into the command line. :D

with the merge of the TMH updates including the forked getid3 library, this fix'll go out with the next train. Yay!

I can start this once this config patch goes out:

Reviving this as we landed the much-redone patch today and it should hit live in the next train. :) Self-assigning as I'll run it myself.

Above patch includes an update to our newly forked copy of getid3 and should include all previously merged but unreleased patches, as well as my additional recent patches for MPEG parsing.

Hm, an alternative is to make [[Media:]] links behave magically via JS in a way that gives you a player on click instead of a raw download. I don't know if that would be a desireable change in behavior though.

IMO [[Media:]] should only be used for raw download links, if at all; for things like inline pronunciation guides we need an inline-triggerable player that actually supports the use case of playing a short audio file on click without replacing the entire tab with a full-window player or a download, depending on your browser and the format.

In the olden days, 'anon' meant 'there's no user_id and no matching user table row, so you can't look them up in the database except for other edits via the same IP which might not be theirs". This definitely does not apply to temporary users, as I understand, who have a user_id and can have their individual contributions looked up.

Looks like a failure that can happen sometimes if no sources are compatible, which appears to be mostly happening on Safari and other WebKit browsers, judging by the user-agents. This should be improved by the patch for T312152 which is still in CR, which cleans up a bunch of config and adds iOS-friendly video output, but it does sound like there's a straight up bug somewhere in that it should either not assume that array is not empty or ensure it isn't. :D

Note I've worked around this in the related cleanup on https://gerrit.wikimedia.org/r/c/mediawiki/extensions/TimedMediaHandler/+/888279 using an explicit -r rate override as necessary based on detected input frame rate being over the max (normally 60).

(More generally, the version of TMH you're running might still have the old dreadful encoding params for VP9 which bias it towards using a high bitrate on complex input -- which is what "noisy" encoding artifacts look like!)

Note that the problem here is that the source file is at *MUCH* too low a bitrate and is thus full of encoding artifacts. The transcode then spends a normal amount of bits replicating all the encoding artifacts as closely as possible.

Whoops, missed the last pings on this one. I suspect preload=meta is likely fine for audio... lemme put this back on my todo list to clear up

Hmm, that file's converting ok for my local test instance with $wgTmhEnableMp4Uploads enabled to upload the .mp4, running in MediaWiki-Docker. The debug log should have the actual command lines and any returned error codes (though.... error codes should cause it to reject the conversion and not save it!)

My suspicion is the job run rate is the culprit, yeah. We really should think about cleaner defaults for this, but the requirement to set up some kind of background service to make the background jobs really work properly is kind of painful. :(

State of the world update!