My question: Suppose a user wants to upload long video (eg. a copyright expired movie), and transcode will likely take hours, if we are implementing transcoding in JavaScript, is it necessary to keep the browser window open? Note that transcoding will also have a high CPU usage (load is slightly larger than 1 on v2c).

I'm unfamiliar with the current Firefogg integration.

Yes, but that is also how Firefogg works.

video2commons already has the video transcode (on the backend instead of frontend) + upload, it wouldn't be impossible to port that to production and integrate with UW.

In T157319#3010421, @zhuyifei1999 wrote:

video2commons already has the video transcode (on the backend instead of frontend) + upload, it wouldn't be impossible to port that to production and integrate with UW.

We'd need to run this through legal to use anything in production; it's already potentially dubious to let an unlicensed h.264 decoder run on labs.

videoconverter.js hasn't seen updates in over 2.5 years. https://github.com/Kagami/ffmpeg.js does the same thing (Emscripten port of ffmpeg) but seems to be maintained.

I've tried it and it took 4:43 hours for ffmpeg.js to convert my test file (compared to 2:41 for ffmpeg binary).
While this was just 1 simple test, I was surprised with the encoding speed. It's not even 2 times slower in the browser.

However, we'd have to ship with an additional 3.5MB script (and maintain it).
And I assume the licensing could be even more of an issue than if we'd host a transcoder somewhere.

The chief problem with ffmpeg.js is that it has the h.264 and AAC decoders built in and shipped to the client -- like in firefogg's embedded copy of ffmpeg -- making it a legal minefield.

I have some theories on how to use the browser's native mp4 playback to capture audio and video frames to feed into the encoder, which might be easier to do by extending ogv.js with encoding than figuring out how to hack a low level API into the ffmpeg.js wrapper. (I've gotten a work branch of WebM encoding working in the iOS native version OGVKit, should be able to adapt it to my JS interfaces. And we're already shipping ogv.js for playback, so maintenance burden may increase less for us.)

I'm also not certain of the performance impact of the JS encoder; ffmpeg.js uses multiple worker threads by default so might have been a bigger per-thread ratio difference versus a local ffmpeg run that was not divided up that way. I'll run some more benchmarks ...

Moving this to the "Product owner backlog" as a potential opportunity more generally across different browsers.

Relevant wishlist item: https://meta.wikimedia.org/wiki/2017_Community_Wishlist_Survey/Multimedia_and_Commons/Upload_and_instant_conversion_of_mpeg_and_avi_etc

While Firefogg seems to be entirely dead now (I couldn’t find anything on firefogg.org about the extension), it’s actually possible to shell out from browser extensions in 2024 – using native messaging, both in Firefox and in Chrome. This requires the user to install two things (the browser extension and the native application), but the latter could bundle ffmpeg, thus avoiding to have to install more things than in the pre-WebExtension times. Having to install an extension still excludes some users (e.g. those who use smartphones), but if it’s more efficient and has less legal risks, it may be worth a try. (And to address the concerns in T157319#3001962, the native app could be written in a way that it continues converting even if the browser tab/window is closed, and the extension part can get the result when the tab is re-opened.)

I suspect in 2024 we're actually better off not using an extension; Web Codecs is deployed in Chrome and Safari and can do hardware-accelerated decoding and encoding of video frames and audio data, leaving only demuxing/muxing to the JavaScript side to do. Firefox is also implementing Web Codecs, and should deploy it at some point.

The main limitation is that only some codecs are supported and may vary between implementations, but the common case is H.264 or HEVC video with AAC-LC audio in some flavor of .mp4 or .mov (same container format, different constraints) and both Chrome and Safari's implementations support VP9 video.

Oh, nice, didn’t know about WebCodecs. If it can be done without an extension (and someone actually does it), that’s definitely better.