Commons uploads: Support automatic conversion of video formats to free formats


As with bug 43149: Allow MPEG and WMV uploads to Commons: transparently convert to the preferred free media format and return that as the final file (and filename extension) on Commons. (For bonus points: support conversion out of RM format as well)

This will add support for the vast majority of all video files in existence, and will allow people who don't understand what "transcode" means from sharing free media with the world.

The Internet Archive provides this service if we don't want to run converters ourselves; we simply have to handle the pipelining to and from IA, who would be happy to see that traffic (and host a copy of the files to boot).

Version: master
Severity: enhancement
See Also:

bzimport added a project: TimedMediaHandler.Via ConduitNov 22 2014, 12:48 AM
bzimport set Reference to bz43150.
Sj created this task.Via LegacyDec 15 2012, 2:32 AM
Dereckson added a comment.Via ConduitDec 15 2012, 2:41 AM

Moving to MediaWiki product, as this is not related to the Commons configuration but to the files handling source code.

Sj added a comment.Via ConduitDec 15 2012, 4:47 AM

(In reply to comment #1)

Moving to MediaWiki product, as this is not related to the Commons
configuration but to the files handling source code.

See my comment on bug 43154 : the most general solution may involve a bit of touching the default file-handling code in MediaWiki, but the simplest solution could just involve changing the Commons-specific file-upload interface.

Bawolff added a comment.Via ConduitDec 15 2012, 9:40 PM

This is a TMH issue. I believe its in the plans for the future. There's probably already a bug for this.

Qgil added a comment.Via ConduitApr 24 2013, 4:20 PM

Just a note to say that user Wywin as submitted a GSoC proposal related to this report:

bzimport added a comment.Via ConduitApr 24 2013, 6:10 PM

mdale wrote:

Note this is supported in TMH:

This is mostly an issue of community feature consensuses. The development effort is mostly complete, involves changing the transcodes configuration option and more testing.

brion added a comment.Via ConduitSep 29 2013, 2:23 PM

Can this be closed out now, or does it need to be repurposed to an 'enable on Wikimedia sites' bug?

Bawolff added a comment.Via ConduitSep 29 2013, 3:02 PM

I think the original request was to convert the file and throw away the original asset, not to transcode the unfree format. (Not sure if that's a good idea, but I don't think that is what tmh does)

brion added a comment.Via ConduitSep 29 2013, 3:18 PM

Throwing away original files is never good... I'd give that a solid wontfix.

bzimport added a comment.Via ConduitSep 29 2013, 4:12 PM

mdale wrote:

Correct. The TMH implementation for auto conversion does encode webm from MPEG-4, it does not throw away or hide originals.

Qgil added a comment.Via ConduitSep 30 2013, 3:53 PM

The request is "Allow MPEG and WMV uploads to Commons: transparently
convert to the preferred free media format and return that as the final file
(and filename extension) on Commons."

The important bit is to allow users to upload videos in non-free formats as opposed to forcing them to do the conversion before uploading.

Kelson added a comment.Via ConduitNov 6 2013, 9:17 AM

Implementing this would certainly be a useful new feature for end users.

Unfortunately, if we do this, we will have to deal with a range of new problems:

  • We work to encourage creation and sharing of free content. This works well with "free/open formats". If we start to support "priopriatery formats", we will somehow fail to encourage the free/open one.
  • We wouldn't store the original file anymore. This would be a switch in the Media handling approach and my feeling: this stinks.
  • If we support "propriatery format/codec", we will have to store a re-encoded version of this file (as original file) and for this reason we will systematicaly loose in quality.
  • Maintenance work will grow. Assuming, we simply use the ffmpeg, this should be relatively easy to maintain. But, if for some reason, ffmpeg has some "legal problems" with "propriatery formats", then we will be impacted too.

All in one, I'm not convinced at all that this would be a good move. IMO, instead of supporting propriatery technologies, we should continue to improve our documentation, help if we can to improve free tools like for example Firefogg.

JeanFred added a comment.Via ConduitNov 11 2013, 6:22 PM

Not sure how much it is relevant, but this came under my radar today:

FFmpeg compiled into JavaScript (using Emscripten). Could theoretically transcode whatever file is thrown by the uploader into WebM.

bzimport added a comment.Via ConduitNov 11 2013, 6:43 PM

mdale wrote:

impressive, is there anything javascript ~can't~ do ;) p.s I could not get the demos to work but I am sure it could work ;)

We have which somewhat similar to this, and integrated into upload wizard, though it runs as a firefox extention with native ffmpeg rather then cross compiled into javascript ;) Which I am sure a wee bit faster.

It not a good user experience to have to encode in browser before the asset is ingested, we lose archival quality, it take a lot longer time for HD content uploads, and we really don't get around the patent issues per say if you're directly hosting this js file for example.

Sj added a comment.Via ConduitSep 25 2014, 8:22 PM

Update: during Wikimania there was discussion among a couple MW devs (thanks, Brion) and Alexis @ the Internet Archive to enable automatic conversion to free formats in something like the following way:

At the upload stage, when detecting a non-free video format

  • Ask for basic title/source information
  • Inform the uploader that [Commons] does not accept that format because it is non-free, and that the Internet Archive will automatically convert it to a free format.
  • Pass the file handle to IA, upload there with the same title & metadata
  • Wait for IA's transcoding to finish, grab the URL of the .webm version
  • Insert that into the TMH upload and continue.

This would address Kelson's concerns, and the concerns raised in the MP4 rfc. IA stores the original, maintains ffmpeg and related code, and handles the conversion. We include a link back to IA in our file description, to attribute its part in the process and point to the original.

Kelson added a comment.Via ConduitSep 26 2014, 1:10 AM

@Sj Interesting new approach. Would be interesting to get a feedback from dev/op people about this approach.

Gilles added a project: Multimedia.Via WebNov 24 2014, 3:20 PM
Aklapper added a project: Commons.Via WebMar 10 2015, 4:45 PM
Steinsplitter moved this task to Uploading on the Commons workboard.Via WebMar 11 2015, 12:55 PM
TheDJ added a subscriber: TheDJ.Via WebMar 11 2015, 1:02 PM

There is now a volunteer developed and maintained service hosted on wmf labs that does all this, with the exception of the transparent part.

JeanFred moved this task to File format support on the Commons workboard.Via WebMar 24 2015, 10:31 AM
Jdforrester-WMF moved this task to Backlog on the Multimedia workboard.Via WebSep 4 2015, 6:13 PM
Restricted Application added subscribers: Steinsplitter, Matanya, Aklapper. · View Herald TranscriptVia HeraldSep 4 2015, 6:13 PM
Jdforrester-WMF removed a project: Multimedia.Via Bulk EditSep 21 2015, 4:03 PM
Sadads added a project: Internet-Archive.Via WebOct 13 2015, 10:16 PM
Sadads added a subscriber: Sadads.
TheDJ moved this task to Format support on the TimedMediaHandler workboard.Via WebOct 25 2015, 2:30 PM
Fuzheado added a subscriber: Fuzheado.Via WebTue, Oct 27, 3:28 PM

FYI, we are now doing this via the Schnittserver and Internet Archive. Not ideal, but at least there is a solution.

Sj added a comment.Via WebTue, Oct 27, 4:14 PM

Super nett. Can you use Schnittserver for mp3 files as well? cf.

Qgil removed a subscriber: Qgil.Via WebThu, Oct 29, 12:41 PM
Nemo_bis added a subscriber: Nemo_bis.Via WebTue, Nov 3, 5:29 PM

There is no need of any special Internet Archive support for this, their conversion facility is open for all. See also

Nemo_bis removed a project: Internet-Archive.Via WebTue, Nov 3, 5:29 PM
Nemo_bis set Security to None.

Add Comment