Page MenuHomePhabricator

brion (Brooke Vibber [old account])
Disabled

Projects (18)

User Details

User Since
Oct 15 2014, 8:30 PM (491 w, 5 d)
Roles
Disabled
IRC Nick
bvibber
LDAP User
Unknown
MediaWiki User
Unknown
This account has been disabled.

Recent Activity

Tue, Feb 20

brion added a comment to T357942: Regression in iOS video playlist output (logic error in refactor).

Fix deployed and I've started a batch job re-running files that may have been affected in the last ~week. Resolved yay \o/

Tue, Feb 20, 10:15 PM · MW-1.42-notes (1.42.0-wmf.19; 2024-02-20), TimedMediaHandler-Transcode
brion updated brion.
Tue, Feb 20, 8:47 PM
brion added a comment to T358044: Migrate dev user accounts for bvibber.

Thanks, in process of creating dev account bvibber ...

Tue, Feb 20, 8:16 PM · Phabricator, SRE, SRE-Access-Requests, LDAP-Access-Requests
brion created T358044: Migrate dev user accounts for bvibber.
Tue, Feb 20, 7:48 PM · Phabricator, SRE, SRE-Access-Requests, LDAP-Access-Requests
brion added a comment to T358026: The "zero" article redirects to enwiki homepage on iOS and Android Wikipedia app .

This smells like PHP seeing "0" and evaluating it as boolean false

Tue, Feb 20, 7:30 PM · Essential-Work, MW-1.42-notes (1.42.0-wmf.20; 2024-02-27), Content-Transform-Team-WIP, Parsoid, Wikipedia-iOS-App-Backlog
brion closed T354502: Videos unplayable on Wikipedia app in iOS/iPadOS 15 as Resolved.

Playlist rewrites ran after the code fix went out... and confirmed, this seems to be fixed now. In iOS 15 the app will show low-res videos, but it will show them now. :)

Tue, Feb 20, 7:29 PM · TimedMediaHandler-Transcode, MW-1.42-notes (1.42.0-wmf.17; 2024-02-06), iOS-app-feature-iPad, Wikipedia-iOS-App-Backlog, VideoJS player
brion claimed T357942: Regression in iOS video playlist output (logic error in refactor).
Tue, Feb 20, 6:52 PM · MW-1.42-notes (1.42.0-wmf.19; 2024-02-20), TimedMediaHandler-Transcode
brion added a comment to T357942: Regression in iOS video playlist output (logic error in refactor).

Scheduled for backport window today, then I'll re-run affected generated tracks

Tue, Feb 20, 6:51 PM · MW-1.42-notes (1.42.0-wmf.19; 2024-02-20), TimedMediaHandler-Transcode

Mon, Feb 19

brion created T357942: Regression in iOS video playlist output (logic error in refactor).
Mon, Feb 19, 11:34 PM · MW-1.42-notes (1.42.0-wmf.19; 2024-02-20), TimedMediaHandler-Transcode

Feb 14 2024

brion closed T357545: estimated file size 3204367 KiB over hard limit 3145728 KiB as Invalid.

We're considering raising the file size limit to 5 GB in T357184, please feel free to follow the technical discussion there.

Feb 14 2024, 7:15 PM · TimedMediaHandler-Transcode, Commons
brion closed T351023: Raise hard limit for transcode sizes to 4 GiB or higher as Invalid.

merging to T357184

Feb 14 2024, 7:12 PM · TimedMediaHandler-Transcode

Feb 13 2024

brion added a comment to T357215: ffmpeg getting killed during transcode.

I had no issue using ffmpeg with this clip on my local machine; could this be an issue in an older version of ffmpeg deployed on WMF servers?

Feb 13 2024, 6:33 PM · TimedMediaHandler-Transcode
brion added a comment to T344378: Spike: How to obtain articles that have images with missing alt text.

Proposed patches to parsoid & extension:linter above add the "missing-image-alt-text" lint checking for missing alt attributes on <img>s with an attached file resource. This is categorized with "low priority" lints and and the data for each match lists a link to the image file in a machine-readable way as well as the positional info in the markup for editing.

Feb 13 2024, 12:01 AM · Wikipedia-iOS-App-Backlog (iOS Release FY2023-24), User-notice, iOS Release FY2023-24 (Archive), Patch-For-Review, Wikipedia-Android-App-Backlog

Feb 12 2024

brion updated subscribers of T357215: ffmpeg getting killed during transcode.

Hm, this could be an issue with the recent changes to the sandboxing of the ffmpeg process. @Joe can you take a peek? This input file is in av1 which may cause it to use more memory for decoding than other files, perhaps, could be tripping up something.

Feb 12 2024, 5:08 PM · TimedMediaHandler-Transcode

Feb 9 2024

brion added a comment to T357184: Consider increasing $wgTranscodeBackgroundSizeLimit to 5GB.

Here's a 4K video that fits in the previous upload limit but has an estimated bitrate resulting in exceeding the 2GB and 3GB limits:

Feb 9 2024, 11:45 PM · TimedMediaHandler, media-backups, SRE-swift-storage, MediaWiki-File-management, Commons

Feb 8 2024

brion added a comment to T357035: webm issue: Some videos on Commons have no duration in their metadata.

Since we aggressively cache file metadata I don't expect parsing the whole file to be significantly expensive here. No compressed data has to be decoded, it'd just be seeking through the file during the getid3() call at upload time, which it may well be doing anyway (I haven't checked the code for mkv/webm, but I know that's exactly what the mpeg parser does to get durations)

Feb 8 2024, 8:05 PM · TimedMediaHandler
brion added a comment to T357035: webm issue: Some videos on Commons have no duration in their metadata.

The funny thing is, if you click on the movie https://commons.wikimedia.org/wiki/File:The_Sea_Beast_(1926).webm to play it, the moment the player comes up, it shows the correct duration. Where does it get that from?

Feb 8 2024, 7:46 PM · TimedMediaHandler
brion added a comment to T357035: webm issue: Some videos on Commons have no duration in their metadata.

ffprobe says it has no listed duration o_O in which case i'm not sure if a duration can be read without parsing the whole file, but getid3 happily does that for mp3 etc ;)

Feb 8 2024, 7:06 PM · TimedMediaHandler
brion added a comment to T357035: webm issue: Some videos on Commons have no duration in their metadata.

This may be another getid3 issue but with the webm/mkv parser, I'll sort through these in a bit and collect a list of affected files to test fixes with.

Feb 8 2024, 7:00 PM · TimedMediaHandler
brion added a project to T357035: webm issue: Some videos on Commons have no duration in their metadata: TimedMediaHandler.
Feb 8 2024, 6:59 PM · TimedMediaHandler
brion updated brion.
Feb 8 2024, 5:26 PM

Feb 7 2024

brion added a comment to T356780: Video transcoding fails when firejail is enabled.

Looks good except that mime type regression, easy fix in gerrit above ^

Feb 7 2024, 7:34 PM · Unstewarded-production-error, MW-1.42-notes (1.42.0-wmf.17; 2024-02-06), TimedMediaHandler-Transcode, Wikimedia-production-error

Feb 6 2024

brion added a comment to T356780: Video transcoding fails when firejail is enabled.

Hm, $wgTranscodeBackgroundMemoryLimit is specified in KiB according to the extension.json doc comment, and the default is 2097152 (2 GiB)

Feb 6 2024, 7:05 PM · Unstewarded-production-error, MW-1.42-notes (1.42.0-wmf.17; 2024-02-06), TimedMediaHandler-Transcode, Wikimedia-production-error

Jan 31 2024

brion added a comment to T354502: Videos unplayable on Wikipedia app in iOS/iPadOS 15.

Proposed fix appears to be working great in local testing; removes the CODECS marker from *only* the 'jpeg' or 'jpeg,mp4a.6b' sources. This might actually get things working on some older iOS versions than 15 too, I'll see what I've got in my test drawer.

Jan 31 2024, 7:41 PM · TimedMediaHandler-Transcode, MW-1.42-notes (1.42.0-wmf.17; 2024-02-06), iOS-app-feature-iPad, Wikipedia-iOS-App-Backlog, VideoJS player
brion added a comment to T354502: Videos unplayable on Wikipedia app in iOS/iPadOS 15.

Obtained a suitable test device (iPhone SE first-gen) which runs iOS 15. Confirmed it will play the MJPEG and MP3 back-compatiblity tracks but it doesn't allow them through the codec filtering in the system HLS player if you list them correctly. Listing a false avc1 codec string works but feels wrong; simply leaving out the "CODECS" key in the playlist seems to be enough to let the HLS player try it on iOS 15, and doesn't appear to interfere with iOS 16 or iOS 17.

Jan 31 2024, 6:31 PM · TimedMediaHandler-Transcode, MW-1.42-notes (1.42.0-wmf.17; 2024-02-06), iOS-app-feature-iPad, Wikipedia-iOS-App-Backlog, VideoJS player

Jan 24 2024

brion added a comment to T330726: Allow extensions to add lint errors.

(cf T344378 for mobile editing microtasks missing alt text planning, we're doing a test spike using linter)

Jan 24 2024, 11:08 PM · MediaWiki-extensions-Linter
brion added a comment to T344378: Spike: How to obtain articles that have images with missing alt text.

Note that linter enthusiasts may have strong opinions about linter data, but there doesn't appear to be a central place to communicate with them. Before deploying any additional lints that go into this API we should ensure we do some proper community discussion to a) warn people it's coming :D b) consider alternatives or mitigations to a possibly very large influx of one type of lint error, and c) improve these processes for the future, so the next time someone wants to add a regular markup check we can do this more easily :D

Jan 24 2024, 11:05 PM · Wikipedia-iOS-App-Backlog (iOS Release FY2023-24), User-notice, iOS Release FY2023-24 (Archive), Patch-For-Review, Wikipedia-Android-App-Backlog
brion added a comment to T330726: Allow extensions to add lint errors.

CCing myself on this as I'm investigating add'l lints meant for editing microtasks :D

Jan 24 2024, 10:47 PM · MediaWiki-extensions-Linter
brion added a comment to T344378: Spike: How to obtain articles that have images with missing alt text.

Looking more into the linter internals; the actual lint checks are done inside Parsoid, in the Linter class; there's no extension/hook interface for adding arbitrary lints (there does appear to be a way for tag extensions to register custom handling, but it won't help us here). Adding either an additional check for missing alt text straight into main should be pretty straightforward I think, and then we can draw from the recorded lints on newly edited pages (and do an offline batch run to prefill them)...

Jan 24 2024, 7:43 PM · Wikipedia-iOS-App-Backlog (iOS Release FY2023-24), User-notice, iOS Release FY2023-24 (Archive), Patch-For-Review, Wikipedia-Android-App-Backlog

Jan 23 2024

brion added a comment to T354502: Videos unplayable on Wikipedia app in iOS/iPadOS 15.

[I suspect what's going on is that iOS 15 doesn't like the HLS packaging after all and some of my compatibility reports were incorrect. We'll confirm this once I get more test devices. In the mobile web the JS automatically detects the failure and fails over to the WebAssembly codec; this isn't done in the app at this time. We can either do the same within the app, if necessary for back-compat, or do an alternate fix of some kind. Keeping this on my claim for this week.]

Jan 23 2024, 7:19 PM · TimedMediaHandler-Transcode, MW-1.42-notes (1.42.0-wmf.17; 2024-02-06), iOS-app-feature-iPad, Wikipedia-iOS-App-Backlog, VideoJS player

Jan 22 2024

brion renamed T354480: WebM with transparency are transcoded with white background from Should force black background on non-transparent video transcodes (was: WebM with transparency are transcoded with white background) to WebM with transparency are transcoded with white background.
Jan 22 2024, 4:55 PM · TimedMediaHandler
brion added a comment to T354480: WebM with transparency are transcoded with white background.

Ok, I'll change it back to the original title though I felt that was not actionable.

Jan 22 2024, 4:55 PM · TimedMediaHandler

Jan 18 2024

brion added a comment to T355292: Port videoscaling to kubernetes.

A harder, but possibly desirable possibility I mentioned on IRC: we could encode each ~10-second input chunk separately, then stitch them back together on completion. This would allow long-duration and high-resolution files to divide up into multiple chunks that can run simultaneously, each on a limited and predictable max core count.

Jan 18 2024, 7:01 PM · Patch-For-Review, Video, TimedMediaHandler, MW-on-K8s, serviceops
brion added a comment to T355292: Port videoscaling to kubernetes.

Another complication on thread count -- the VP9 encoder can only make use of so many threads effectively, based on the size of the frame (controls number of macroblocks that can be simultaneously run) :D So usage is tough to predict from the job type. We *could* implement some kind of variable size thing where we "block up" so a 240p takes 2 cpus and a 2160 uses 8 cpus etc but I don't know if that can be expressed sensibly.

Jan 18 2024, 6:53 PM · Patch-For-Review, Video, TimedMediaHandler, MW-on-K8s, serviceops
brion added a comment to T355292: Port videoscaling to kubernetes.

Couple quick notes:

  • Reducing thread count is IMO a very bad idea, as most of the time there will be few jobs and they may be high resolution videos. You want to use the maximum number of available threads to keep CPUs occupied or else you're going to waste a lot of time and make the jobs run a lot slower at high resolutions (the slowest jobs). It would be better to sometimes be at full load on a server than to have a single job that takes days instead of hours or hours instead of minutes.
  • The task roughly checks out a source file, decides an ffmpeg command line or two, runs them and then stores and processes the resulting output file. There are two ffmpeg passes at present for better bitrate control but if need be a single pass command can be used instead. This may run up to several hours for very large or long videos -- thus this is where we have the potential to split the job into three components:
  • regular media wiki job gets the info and builds the command options and sends that to a script or service
  • which transcodes the file according to given command settings, without media wiki logic itself and calls back to mediawiki API which queues a
  • cleanup job to import the file and update database state
Jan 18 2024, 3:11 PM · Patch-For-Review, Video, TimedMediaHandler, MW-on-K8s, serviceops

Jan 17 2024

brion added a comment to T355264: In the MediaSearch impossible to play video and there is no preview.

I'm currently blocked on CirrusSearch but I can fix this from the TMH side with a quick hack. We'll bring the underlying internal APIs back up in a bit once I can test my changes. :D

Jan 17 2024, 10:44 PM · MW-1.42-notes (1.42.0-wmf.15; 2024-01-23), SDAW-MediaSearch, Structured-Data-Backlog, VideoJS player
brion placed T299291: Improve videoJS MediaSearch integration up for grabs.

Currently blocked on setting up a local CirrusSearch to drive MediaSearch for local testing. For now I'll restore the original namespacing so the old code should continue to work, but I can't test it.

Jan 17 2024, 10:43 PM · SDAW-MediaSearch, Structured-Data-Backlog, VideoJS player
brion claimed T299291: Improve videoJS MediaSearch integration.

I'm cleaning this code up right now :D We had a regression due to internal refactoring breaking the way the APIs were referenced. I think I can remove the internals reference and make it cleaner.

Jan 17 2024, 10:24 PM · SDAW-MediaSearch, Structured-Data-Backlog, VideoJS player
brion added a comment to T355264: In the MediaSearch impossible to play video and there is no preview.

Oh wait it's not from there, it's from MediaSearch's invocation. :D Ok I see; we had a regression due to removing OgvJsSupport from the mw namespace in the ES6 refactoring; MediaSearch was directly invoking it and videojs. I'll clean this up on the MediaSearch side.

Jan 17 2024, 10:01 PM · MW-1.42-notes (1.42.0-wmf.15; 2024-01-23), SDAW-MediaSearch, Structured-Data-Backlog, VideoJS player
brion claimed T355264: In the MediaSearch impossible to play video and there is no preview.

This in ext.tmh.player.inline.js is failing:

Jan 17 2024, 9:43 PM · MW-1.42-notes (1.42.0-wmf.15; 2024-01-23), SDAW-MediaSearch, Structured-Data-Backlog, VideoJS player

Jan 16 2024

brion added a comment to T157319: Implement replacement for Firefogg (on-the-fly video transcoding).

I suspect in 2024 we're actually better off not using an extension; Web Codecs is deployed in Chrome and Safari and can do hardware-accelerated decoding and encoding of video frames and audio data, leaving only demuxing/muxing to the JavaScript side to do. Firefox is also implementing Web Codecs, and should deploy it at some point.

Jan 16 2024, 8:00 PM · Video, Multimedia, UploadWizard

Jan 8 2024

brion added a comment to T354490: Option to force lossless, non-resampled audio format conversion (was: Disabling Automated Transcodes).

I had some FLAC issue on Firefox in the past, but we can use flac's if it's supported right now (only 16bit, using flac in 8bit is not good because filesizes are going to be same).

Jan 8 2024, 11:40 PM · TimedMediaHandler-Transcode, Commons
brion added a comment to T354490: Option to force lossless, non-resampled audio format conversion (was: Disabling Automated Transcodes).

caniuse claims same current-version support for both:

Jan 8 2024, 11:26 PM · TimedMediaHandler-Transcode, Commons
brion added a comment to T354490: Option to force lossless, non-resampled audio format conversion (was: Disabling Automated Transcodes).

just using the wav source file directly, something like this:

Jan 8 2024, 11:23 PM · TimedMediaHandler-Transcode, Commons
brion added a comment to T354490: Option to force lossless, non-resampled audio format conversion (was: Disabling Automated Transcodes).

(Note: if the proposed fix for this is not to use WAV pass-through or FLAC transcode, this will probably simply never be done and we should just close.)

Jan 8 2024, 11:20 PM · TimedMediaHandler-Transcode, Commons
brion added a comment to T354490: Option to force lossless, non-resampled audio format conversion (was: Disabling Automated Transcodes).

Just confirming terms. ;) Duration is always about time. File size is pretty much irrelevant; these are niche files with small audiences, and most likely all relevant files will be fairly short.

Jan 8 2024, 11:18 PM · TimedMediaHandler-Transcode, Commons
brion added a comment to T354490: Option to force lossless, non-resampled audio format conversion (was: Disabling Automated Transcodes).

About resampling: Resampling is just the same thing with slowing down or speeding up something right? I think they are same things but if you think different: Transcodes must not be resampled to don't destroy their structure. Sometimes sampling rates will be incompatible, if that's happened (for example, if samplerate is 10000): Transcodes must be generated in nearest supported rate (11025 in this case) and contain the smilar rawdata. So, the MP3 will be sound like the same speed when it's played on 10000 so a little bit slower.

Jan 8 2024, 10:57 PM · TimedMediaHandler-Transcode, Commons
brion added a comment to T354490: Option to force lossless, non-resampled audio format conversion (was: Disabling Automated Transcodes).

Ok, changing the title again to clarify what I originally understood was mostly correct: you want a way to avoid resampling, and there is *nothing specific wrong with the transcodes* except that they're resampled/not-lossless in the first place. :)

Jan 8 2024, 10:48 PM · TimedMediaHandler-Transcode, Commons
brion renamed T354490: Option to force lossless, non-resampled audio format conversion (was: Disabling Automated Transcodes) from Incorrect transcodes of specific WAV audio files (was: Disabling Automated Transcodes) to Option to force lossless, non-resampled audio format conversion (was: Disabling Automated Transcodes).
Jan 8 2024, 10:32 PM · TimedMediaHandler-Transcode, Commons
brion added a comment to T354490: Option to force lossless, non-resampled audio format conversion (was: Disabling Automated Transcodes).

(I'm wondering also if your .wav files might be themselves incorrect to begin with and you're just not playing them back to check them against the MP3 or Ogg Vorbis transcodes?)

Jan 8 2024, 10:19 PM · TimedMediaHandler-Transcode, Commons
brion added a comment to T354490: Option to force lossless, non-resampled audio format conversion (was: Disabling Automated Transcodes).

1: They are slowed down extremely, sounds like in the same speed in 44100Hz. Transcodes must not be slowed down.

Jan 8 2024, 10:13 PM · TimedMediaHandler-Transcode, Commons
brion added a comment to T354490: Option to force lossless, non-resampled audio format conversion (was: Disabling Automated Transcodes).

I cannot reproduce any difference in speed between these three files as played back in VLC or in Chrome:

Jan 8 2024, 9:44 PM · TimedMediaHandler-Transcode, Commons
brion claimed T354490: Option to force lossless, non-resampled audio format conversion (was: Disabling Automated Transcodes).

This task is absolutely not for increasing quality. Transcodes are totally different from sources. They are slowed down (or speeded up) extremely in most of files.

Jan 8 2024, 9:39 PM · TimedMediaHandler-Transcode, Commons
brion renamed T354490: Option to force lossless, non-resampled audio format conversion (was: Disabling Automated Transcodes) from Avoid resampling/quality loss in audio transcodes (was: Disabling Automated Transcodes) to Incorrect transcodes of specific WAV audio files (was: Disabling Automated Transcodes).
Jan 8 2024, 9:37 PM · TimedMediaHandler-Transcode, Commons
brion added a comment to T354480: WebM with transparency are transcoded with white background.

[Note that fuller support for transparent videos would be _neat_ but likely only useful in combination with autoplay animations. We can think about that later, if you're interested in it happening let's open a separate feature request and we'll throw around ideas about how to fall back for non-transparent-video-supporting browsers.]

Jan 8 2024, 6:02 PM · TimedMediaHandler
brion renamed T354480: WebM with transparency are transcoded with white background from WebM with transparency are transcoded with white background to Should force black background on non-transparent video transcodes (was: WebM with transparency are transcoded with white background).
Jan 8 2024, 6:00 PM · TimedMediaHandler
brion closed T200873: vp9 transcode failures with signal 15 as Resolved.

[old, not currently occurring, ran old items long ago, closing out]

Jan 8 2024, 5:58 PM · TimedMediaHandler-Transcode
brion added a comment to T354480: WebM with transparency are transcoded with white background.

Note also that VP9 transparency is not supported in Safari.

Jan 8 2024, 5:54 PM · TimedMediaHandler
brion added a comment to T354490: Option to force lossless, non-resampled audio format conversion (was: Disabling Automated Transcodes).

Thanks, I'll download and test those this afternoon. :)

Jan 8 2024, 5:44 PM · TimedMediaHandler-Transcode, Commons
brion added a comment to T354490: Option to force lossless, non-resampled audio format conversion (was: Disabling Automated Transcodes).

Note if anything is being "slowed down" this is a SERIOUS FUNCTIONAL ERROR THAT SHOULD NOT BE HAPPENING; please report that specific file to me with a link in a separate bug report.

Jan 8 2024, 3:52 PM · TimedMediaHandler-Transcode, Commons
brion renamed T354490: Option to force lossless, non-resampled audio format conversion (was: Disabling Automated Transcodes) from Disabling Automatic Transcodes to Avoid resampling/quality loss in audio transcodes (was: Disabling Automated Transcodes).
Jan 8 2024, 3:50 PM · TimedMediaHandler-Transcode, Commons
brion added a comment to T354490: Option to force lossless, non-resampled audio format conversion (was: Disabling Automated Transcodes).

This is a low-priority feature request. It's not a bad request, but it's low priority because it's a cleanup/nice-to-have, not a functional issue. Whether it gets gotten to in the next few weeks or not is based more on how easy it is to implement cleanly without drawbacks than how important it is. :)

Jan 8 2024, 3:47 PM · TimedMediaHandler-Transcode, Commons
brion added a comment to T354502: Videos unplayable on Wikipedia app in iOS/iPadOS 15.

This was working last month in tests on git build, I'll double check this week

Jan 8 2024, 4:13 AM · TimedMediaHandler-Transcode, MW-1.42-notes (1.42.0-wmf.17; 2024-02-06), iOS-app-feature-iPad, Wikipedia-iOS-App-Backlog, VideoJS player
brion claimed T354502: Videos unplayable on Wikipedia app in iOS/iPadOS 15.
Jan 8 2024, 4:13 AM · TimedMediaHandler-Transcode, MW-1.42-notes (1.42.0-wmf.17; 2024-02-06), iOS-app-feature-iPad, Wikipedia-iOS-App-Backlog, VideoJS player
brion claimed T354501: Videos unplayable on Safari via Apple devices running iOS 11–15.

Should work on iOS 12 and up iirc, I'll have to run more tests

Jan 8 2024, 4:11 AM · TimedMediaHandler-Transcode, Browser-Support-Apple-Safari, Mobile
brion added a comment to T68722: [iOS app] Some media (esp. video) files do not work.

I can't help wonder the issue with VideoJS player has resolved. I created tasks T354501 and T354502 about VideoJS player on older iOS versions, but are they related to this task?

Jan 8 2024, 4:10 AM · MW-1.42-notes (1.42.0-wmf.7; 2023-11-28), Patch-For-Review, Essential-Work, Wikipedia-iOS-App-Backlog

Dec 12 2023

brion claimed T353163: Creation of dynamic property File_Ogg_Theora::$_firstGranulePos is deprecated.
Dec 12 2023, 4:46 PM · MW-1.42-notes (1.42.0-wmf.15; 2024-01-23), PHP 8.2 support, TimedMediaHandler

Dec 4 2023

brion added a comment to T68722: [iOS app] Some media (esp. video) files do not work.

BTW. I realised that HLS doesn't support media temporal fragments, so likely start and end params won't work on iOS. There is probably no easy way around that, we'll likely have to come up with something custom.

m3u8 has #EXT-X-START:TIME-OFFSET=-15, but as start/end are transclusion params, we can't easily embed them into the manifest without using dynamic manifests (currently we have static manifests)

Dec 4 2023, 11:11 PM · MW-1.42-notes (1.42.0-wmf.7; 2023-11-28), Patch-For-Review, Essential-Work, Wikipedia-iOS-App-Backlog
brion added a comment to T68722: [iOS app] Some media (esp. video) files do not work.

Started bulk re-encoding with the remuxing being used transparently whenever possible. I expect full coverage at at least base resolution by end of calendar year. :D

Dec 4 2023, 11:05 PM · MW-1.42-notes (1.42.0-wmf.7; 2023-11-28), Patch-For-Review, Essential-Work, Wikipedia-iOS-App-Backlog

Nov 21 2023

brion closed T350996: HLS meta playlist .m3u8 includes not-yet-made transcodes as Resolved.

Closing.

Nov 21 2023, 8:52 PM · MW-1.42-notes (1.42.0-wmf.4; 2023-11-07), TimedMediaHandler-Transcode

Nov 20 2023

brion added a comment to T351557: Transcode in "Unknown status" and "reset" or purge does not help.

[In general I know we've got some order of operations problems with the transcode table and other db bits, the job queue, and the file storage -- three distinct storage systems which need to be either in sync or in the proper order to handle weird sync issues. I'm going to collect up a few of these related issues today and try to work them out in the next couple days. If we're lucky I'll have something up for code review before the Thanksgiving holiday Thursday! If not I'll keep poking at it later.]

Nov 20 2023, 5:15 PM · Commons, TimedMediaHandler-Transcode
brion claimed T351557: Transcode in "Unknown status" and "reset" or purge does not help.

I'll add this to my queue. Currently I'm chasing down bulk encodings; chasing down ongoing errors will be next and I expect several synchronization-related problems with cached data chunks. :)

Nov 20 2023, 4:41 PM · Commons, TimedMediaHandler-Transcode

Nov 19 2023

brion changed the status of T350996: HLS meta playlist .m3u8 includes not-yet-made transcodes from Open to In Progress.

Script completed. If no surprises during the next couple days of rebuilds, all should be well to resolve. :D

Nov 19 2023, 2:08 AM · MW-1.42-notes (1.42.0-wmf.4; 2023-11-07), TimedMediaHandler-Transcode

Nov 17 2023

brion added a comment to T350996: HLS meta playlist .m3u8 includes not-yet-made transcodes.

Script is up to the "An"s, I expect it to complete within ~24-48 hours.

Nov 17 2023, 10:05 PM · MW-1.42-notes (1.42.0-wmf.4; 2023-11-07), TimedMediaHandler-Transcode
brion reopened T350996: HLS meta playlist .m3u8 includes not-yet-made transcodes as "Open".

Running a one-off rebuild of streaming manifests separately from the active transcodes. This should fix any that were damaged that haven't had a chance to be fixed by other runs yet:

Nov 17 2023, 7:52 PM · MW-1.42-notes (1.42.0-wmf.4; 2023-11-07), TimedMediaHandler-Transcode
brion added a comment to T350996: HLS meta playlist .m3u8 includes not-yet-made transcodes.

It'll require actually updating one of the other transcodes on an affected file to rebuild the m3u8 to work again... I've got several batches running, but I might be able to force a simple regeneration too.

Nov 17 2023, 7:20 PM · MW-1.42-notes (1.42.0-wmf.4; 2023-11-07), TimedMediaHandler-Transcode

Nov 15 2023

brion closed T350996: HLS meta playlist .m3u8 includes not-yet-made transcodes as Resolved.

Fix deployed, these'll get cleaned up as transcodes run.

Nov 15 2023, 10:43 PM · MW-1.42-notes (1.42.0-wmf.4; 2023-11-07), TimedMediaHandler-Transcode

Nov 13 2023

Krinkle awarded T152851: TMH should not make DB writes on HTTP GET for its on-the-fly transcode corrections a Love token.
Nov 13 2023, 5:30 PM · MW-1.42-notes (1.42.0-wmf.4; 2023-11-07), Wikimedia-Performance-recommendation, Wikimedia-production-error, TimedMediaHandler
brion added a comment to T152851: TMH should not make DB writes on HTTP GET for its on-the-fly transcode corrections.

I think we can just take this bit out safely, it wasn't reliable and the requeue scripts have to handle the case where a row just got left anyway. :D Patch coming...

Nov 13 2023, 4:35 PM · MW-1.42-notes (1.42.0-wmf.4; 2023-11-07), Wikimedia-Performance-recommendation, Wikimedia-production-error, TimedMediaHandler
brion claimed T152851: TMH should not make DB writes on HTTP GET for its on-the-fly transcode corrections.

Oof, yeah that explains it. I'll add it to my pile of fixes for this week. :D

Nov 13 2023, 4:24 PM · MW-1.42-notes (1.42.0-wmf.4; 2023-11-07), Wikimedia-Performance-recommendation, Wikimedia-production-error, TimedMediaHandler

Nov 12 2023

brion added a comment to T152851: TMH should not make DB writes on HTTP GET for its on-the-fly transcode corrections.

Nothing should be writing to the database from TimedMediaHandler in page views, as far as I know. I think there used to be something years ago but if it's still there it needs to be removed. Is this still a problem that anyone is aware of?

Nov 12 2023, 1:54 AM · MW-1.42-notes (1.42.0-wmf.4; 2023-11-07), Wikimedia-Performance-recommendation, Wikimedia-production-error, TimedMediaHandler
brion closed T309823: Disable old WebM VP8 transcodes except for 360p as Resolved.

done

Nov 12 2023, 1:48 AM · Wikimedia-Site-requests
brion closed T312153: Batch run of TMH requeueTranscodes to remove now-unused 120p and 180p low-res files as Resolved.

this finished :D

Nov 12 2023, 1:47 AM · Wikimedia-maintenance-script-run, TimedMediaHandler-Transcode
brion closed T312153: Batch run of TMH requeueTranscodes to remove now-unused 120p and 180p low-res files, a subtask of T312152: Clean up video transcode config for speed/bitrate balance, as Resolved.
Nov 12 2023, 1:46 AM · MW-1.39-notes (1.39.0-wmf.27; 2022-08-29), TimedMediaHandler-Transcode
brion added a comment to T200939: Intermittent transcode error "Invalid frame size: 0x0.".

This still happens, and it's mildly annoying. ;) We may need to force proper chronology on the database connections used to fetch file data? Anyway, back on the slate for cleanup work :D

Nov 12 2023, 12:21 AM · MW-1.42-notes (1.42.0-wmf.7; 2023-11-28), User-TheDJ, TimedMediaHandler-Transcode

Nov 11 2023

brion closed T104062: Halve frame rate on 48/50/60fps video transcodes as Declined.

I changed my mind and, in recent updates, instead added extra bandwidth for the higher-frame-rate videos so they still compress nicely.

Nov 11 2023, 9:22 PM · TimedMediaHandler-Transcode
brion added a subtask for T166025: Enable AAC-LC support for Commons uploads and TMH/etc. playback (not patent-encumbered anymore since 2018): T155320: Implement strict mime type detection and media type inferring of audio/video files.
Nov 11 2023, 9:16 PM · TimedMediaHandler-Transcode, UploadWizard, WMF-Legal, Commons, Multimedia
brion added a subtask for T159885: Support upload of MJPEG video in AVI or MOV container: T155320: Implement strict mime type detection and media type inferring of audio/video files.
Nov 11 2023, 9:16 PM · TimedMediaHandler-Transcode, Video
brion added a subtask for T329258: Enable H.264/MP4 support for Commons uploads and TMH/etc. playback : T155320: Implement strict mime type detection and media type inferring of audio/video files.
Nov 11 2023, 9:16 PM · TimedMediaHandler-Transcode, Commons
brion added parent tasks for T155320: Implement strict mime type detection and media type inferring of audio/video files: T166025: Enable AAC-LC support for Commons uploads and TMH/etc. playback (not patent-encumbered anymore since 2018), T159885: Support upload of MJPEG video in AVI or MOV container, T329258: Enable H.264/MP4 support for Commons uploads and TMH/etc. playback .
Nov 11 2023, 9:16 PM · TimedMediaHandler-Transcode, Commons, MediaWiki-File-management, Multimedia, Technical-Debt
brion added a comment to T159885: Support upload of MJPEG video in AVI or MOV container.

For .mov container input, which is reasonably common on certain generations of cameras, all we should need now is consistent support for ISO BMFF-family input files with proper codec validation (right now we kinda half-ass it). Depending on source files, some may have AAC audio as well (cf T166025).

Nov 11 2023, 9:12 PM · TimedMediaHandler-Transcode, Video
brion reopened T304624: WebM with multichannel OPUS audio fails on Safari as "Open".

Hm, maybe I misremember. I'll check this back next week, it still isn't playing in Safari and I'm not sure if it's a cache thing or a breakage thing and I'll deal with it Monday.

Nov 11 2023, 8:19 PM · Browser-Support-Apple-Safari, MW-1.39-notes (1.39.0-wmf.8; 2022-04-18), TimedMediaHandler
brion closed T304624: WebM with multichannel OPUS audio fails on Safari as Resolved.

I've implemented the squooshing to 2-channel output in recent updates since We Can't Have Nice Things. Re-running transcodes on this file. See T351025 for the more permanent long-term solution of adding surround tracks probably using AAC

Nov 11 2023, 7:36 PM · Browser-Support-Apple-Safari, MW-1.39-notes (1.39.0-wmf.8; 2022-04-18), TimedMediaHandler
brion added a parent task for T166025: Enable AAC-LC support for Commons uploads and TMH/etc. playback (not patent-encumbered anymore since 2018): T351025: Multichannel surround sound track for video transcodes (at least 5.1-channel support).
Nov 11 2023, 7:34 PM · TimedMediaHandler-Transcode, UploadWizard, WMF-Legal, Commons, Multimedia
brion added a subtask for T351025: Multichannel surround sound track for video transcodes (at least 5.1-channel support): T166025: Enable AAC-LC support for Commons uploads and TMH/etc. playback (not patent-encumbered anymore since 2018).
Nov 11 2023, 7:34 PM · TimedMediaHandler-Transcode
brion created T351025: Multichannel surround sound track for video transcodes (at least 5.1-channel support).
Nov 11 2023, 7:33 PM · TimedMediaHandler-Transcode
brion added a comment to T166025: Enable AAC-LC support for Commons uploads and TMH/etc. playback (not patent-encumbered anymore since 2018).

Current state of this:

Nov 11 2023, 7:24 PM · TimedMediaHandler-Transcode, UploadWizard, WMF-Legal, Commons, Multimedia
brion created T351023: Raise hard limit for transcode sizes to 4 GiB or higher.
Nov 11 2023, 6:48 PM · TimedMediaHandler-Transcode

Nov 10 2023

brion created T350996: HLS meta playlist .m3u8 includes not-yet-made transcodes.
Nov 10 2023, 4:58 PM · MW-1.42-notes (1.42.0-wmf.4; 2023-11-07), TimedMediaHandler-Transcode

Nov 8 2023

brion created T350816: Special:Transcode_statistics is very slow to load on Commons, cuts off lots of info.
Nov 8 2023, 6:00 PM · MW-1.42-notes (1.42.0-wmf.19; 2024-02-20), TimedMediaHandler