Page MenuHomePhabricator

Once it's not patent-encumbered, enable MPEG-2 support for Commons uploads and TMH/etc. playback
Closed, ResolvedPublic

Description

Apparently, the last patent will expire 14 February 2018: List of United States MPEG-2 patents

Putting this up so legal has enough time. And that they don't duplicate work for the audio layer T120288.

What uses MPEG-2? DVD-Video and OTA broadcasts in the US.

Related Objects

Event Timeline

Poyekhali changed the task status from Open to Stalled.May 22 2017, 2:16 AM
Poyekhali triaged this task as Lowest priority.

There are already very few patents remaining on MPEG-2. It is probably not as important as MP3 though.

According to MPEG-LA, MPEG-2 is patent free in the United States now.

This task first needs to answer, in which file format it wants to support the codec. Codec detection happens in the file format parsing, so for each file format you want the codec to be supported in, you will have to take separate technical steps on that level (and thus likely separate tickets).

So I think this is gonna be MPEG-2 transport stream or something? I'm a little rusty. Lemme do a quick spike with samples from Internet Archive and see what they look like.

Change 411051 had a related patch set uploaded (by Brion VIBBER; owner: Brion VIBBER):
[mediawiki/extensions/TimedMediaHandler@master] WIP: Support MPEG-1/MPEG-2 video uploads

https://gerrit.wikimedia.org/r/411051

brion changed the task status from Stalled to Open.Feb 15 2018, 6:43 PM

I fixed the aspect ratio problem I originally saw, but need to figure out how to detect interlacing and then apply deinterlacing on the transcodes...

Now detects interlaced input and applies yadif deinterlacing filter on transcodes.

So far so good with the spike patch, except that some files I've tested don't detect the audio track, possibly some incomplete support in getid3. It also returns the wrong pixel aspect ratio for MPEG-2 video, which I was able to work around on the TMH end for now.

(Note that I believe standard .mpg and .mpeg files are in fact program streams, not transport streams, which is used for broadcast and some of the streams for Apple's HLS.)

Most browsers don't play MPEG-2 in <video> tags (Safari on Mac does, but not on iOS) and ogv.js doesn't yet include MPEG playback, so during the time between initial upload and the completion of the first transcode, users may see an ugly message about unsupported formats instead of it trying to play back the original source. Will want eventually to rethink the UI around transcoding status reporting on the File: page!

Question: is there any concern about extra patrolling needs for piracy in the case of decrypted DVD rips? Or would the large original size of full-length movies in MPEG-2 make this too difficult for drive-by uploaders to exploit?

If you are concerned of DVD rips, these used to be either the full .iso's rips, or they were the VIDEO_TS folders within them. VIDEO_TS is a structure around mpeg-ps files that includes the DVD metadata etc. BTW. DVDs are kinda special, if you'd play their raw files then likely you wouldn't enjoy the movie much, as they are divided in chapters, and sometimes this is quite messy. You really actually need that DVD specific metadata.

The most common format is MPEG-TS these days (used for DVB-C, DVB-T and DVB-S). MPEG-TS is a super flexible, timing accurate and streaming capable file format, that can basically include whatever you want. It supports hundreds of different type of codecs and metadata (EPG, chapters, menu's, subtitles, teletext etc).

I always say that MPEG-TS is what satellite engineers understood about writing specifications, that the Internet part of the world only understood just a few years ago. It's as flexible as mov/mp4/mkv, but with slight overhead, simply better manageable for the hardware industry.

Honestly, MPEG-PS is rare. It exists, but not nearly as much as you'd expect. DVDs are a super spec on top of it, or they are transcoded to simpler codecs, Broadcast video is MPEG-TS or ripped to other formats, and archived material is either the original captured video in camera specific formats, MXF or proprietary stuff around MPEG-TS.

I wonder if there is much matroska wrapped mpeg2 video...

I didn't have any MPEG-TS files handy but I did a quick test creating one with ffmpeg -- ffmpeg will take it as input and convert it just fine, but getid3's handling is incomplete and it errors out.

To support TS as well as PS, would need:

  • complete the logic in getid3's module.audio-video.ts.php and patch it upstream
    • (or replace with shell-out to ffprobe)
  • make sure the mime detection maps .ts/.tsv/.tsa files to video/mp2t and does the correct magic check
  • route the file handler class appropriately
  • validate codecs so we don't let streams with MPEG-4, H.264 etc through

Note that for WMF production I think we'd also need to change our thumbor configuration to make sure the files get passed through the ffmpeg handler:

https://phabricator.wikimedia.org/diffusion/THMBREXT/browse/master/wikimedia_thumbor/loader/video/__init__.py

I don't think anything special would need to be added other than adding checks for the '.mpg' and '.mpeg' extensions.

Change 411051 had a related patch set uploaded (by Brion VIBBER; owner: Brion VIBBER):
[mediawiki/extensions/TimedMediaHandler@master] Support MPEG-1/MPEG-2 video uploads

https://gerrit.wikimedia.org/r/411051

Change 411051 merged by jenkins-bot:
[mediawiki/extensions/TimedMediaHandler@master] Support MPEG-1/MPEG-2 video uploads

https://gerrit.wikimedia.org/r/411051

Roughly planning to soft-launch this and try replacing some of the manual .ogv conversions of old mpeg1 and 2 files from scientific papers with the originals. I'll track that on a separate task once the release train brings it live.

A couple things that could be improved later:

  • Better playback error message when no transcodes are available yet
  • Make sure thumbnails get deinterlaced where applicable
  • More thorough tests for type detection and track info detection

Change 569341 had a related patch set uploaded (by Brion VIBBER; owner: Brion VIBBER):
[operations/software/thumbor-plugins@master] Support MPEG-1 and MPEG-2 video files with .mpg or .mpeg extension

https://gerrit.wikimedia.org/r/569341

Change 617874 had a related patch set uploaded (by AntiCompositeNumber; owner: AntiCompositeNumber):
[operations/software/thumbor-plugins@master] Add tests for MPEG-1 and MPEG-2 thumbnailing

https://gerrit.wikimedia.org/r/617874

Change 569341 merged by jenkins-bot:
[operations/software/thumbor-plugins@master] Support MPEG-1 and MPEG-2 video files with .mpg or .mpeg extension

https://gerrit.wikimedia.org/r/569341

Change 617874 merged by Gilles:
[operations/software/thumbor-plugins@master] Add tests for MPEG-1 and MPEG-2 thumbnailing

https://gerrit.wikimedia.org/r/617874

TheDJ claimed this task.

I think we should close this, as we now allow uploads of mpeg1 and mpeg2 video, audio and programs streams.

I'll open a separate ticket for the MPEG-TS file format.