Page MenuHomePhabricator

Implement replacement for Firefogg (on-the-fly video transcoding)
Open, Needs TriagePublic

Description

For policy reasons, Wikimedia Commons doesn't want to host videos using proprietary file formats, like MP4 (https://commons.wikimedia.org/wiki/Commons:Requests_for_comment/MP4_Video). UploadWizard used to use the Firefogg extension (https://firefogg.org/) for on-the-fly transcoding of MP4 files to OGG in the user's browser. But due to changes to how Firefox handles extensions, this is soon to stop working (https://blog.mozilla.org/addons/2016/11/23/add-ons-in-2017/) and it seems already impossible to install the extension.

With modern browsers' fast JavaScript engines and HTML5 file APIs, it should totally be possible to implement transcoding in JavaScript, without shelling out to external executables, like Firefogg did. As a bonus, that would work on more browsers than Firefox.

Event Timeline

matmarex created this task.Feb 6 2017, 2:03 PM
Restricted Application added a project: Multimedia. · View Herald TranscriptFeb 6 2017, 2:03 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

My question: Suppose a user wants to upload long video (eg. a copyright expired movie), and transcode will likely take hours, if we are implementing transcoding in JavaScript, is it necessary to keep the browser window open? Note that transcoding will also have a high CPU usage (load is slightly larger than 1 on v2c).

I'm unfamiliar with the current Firefogg integration.

Yes, but that is also how Firefogg works.

TheDJ added a subscriber: TheDJ.Feb 6 2017, 3:53 PM
[18:46]	<MatmaRex> also, is anyone looking into implementing a replacement [for Firefogg]? we could integrate something like http://bgrins.github.io/videoconverter.js/demo/ (if you google 'ffmpeg js', this is the top result, but there's like ten other libraries that do the same thing in slightly different ways. surely one of them will fit our needs)
[18:49]	<marktraceur> I wonder if brion has any thoughts on that.
[18:50]	<brion> MatmaRex: ffmpeg.js is pretty slow but does work in a pinch :)
[18:50]	<brion> i think we might get a better experience from a native app converter+uploader but that's work too
[18:50]	<brion> and/or i can keep working on intermediary conversion :)
[18:51]	<brion> oh yeah one other problem with those: legal (ffmpeg JS conversion)
[18:51]	<brion> since it compiles and ships an h.264 decoder
[18:51]	<marktraceur> brion: I figured that might be a thing
[18:51]	<MatmaRex> brion: right. i'm just hoping to retain some no-fuss option in uploadwizard for non-savvy users, to replace firefogg.
[18:52]	<brion> yeah i think i'm just going to have to finish the intermediary stuff, shipping out to toollabs or archive.org for conversion
[18:52]	<marktraceur> That seem slike a good option, yeah
[18:52]	<brion> i suspect i can make that integrate with uploadwizard but am prototyping separate
Legoktm added a subscriber: Legoktm.Feb 8 2017, 7:20 PM

video2commons already has the video transcode (on the backend instead of frontend) + upload, it wouldn't be impossible to port that to production and integrate with UW.

brion added a comment.Feb 8 2017, 7:23 PM

video2commons already has the video transcode (on the backend instead of frontend) + upload, it wouldn't be impossible to port that to production and integrate with UW.

We'd need to run this through legal to use anything in production; it's already potentially dubious to let an unlicensed h.264 decoder run on labs.

videoconverter.js hasn't seen updates in over 2.5 years. https://github.com/Kagami/ffmpeg.js does the same thing (Emscripten port of ffmpeg) but seems to be maintained.

I've tried it and it took 4:43 hours for ffmpeg.js to convert my test file (compared to 2:41 for ffmpeg binary).
While this was just 1 simple test, I was surprised with the encoding speed. It's not even 2 times slower in the browser.

However, we'd have to ship with an additional 3.5MB script (and maintain it).
And I assume the licensing could be even more of an issue than if we'd host a transcoder somewhere.

The chief problem with ffmpeg.js is that it has the h.264 and AAC decoders built in and shipped to the client -- like in firefogg's embedded copy of ffmpeg -- making it a legal minefield.

I have some theories on how to use the browser's native mp4 playback to capture audio and video frames to feed into the encoder, which might be easier to do by extending ogv.js with encoding than figuring out how to hack a low level API into the ffmpeg.js wrapper. (I've gotten a work branch of WebM encoding working in the iOS native version OGVKit, should be able to adapt it to my JS interfaces. And we're already shipping ogv.js for playback, so maintenance burden may increase less for us.)

I'm also not certain of the performance impact of the JS encoder; ffmpeg.js uses multiple worker threads by default so might have been a bigger per-thread ratio difference versus a local ffmpeg run that was not divided up that way. I'll run some more benchmarks ...

dr0ptp4kt added a subscriber: dr0ptp4kt.

Moving this to the "Product owner backlog" as a potential opportunity more generally across different browsers.

Dvorapa removed a subscriber: Dvorapa.