Page MenuHomePhabricator

Mysterious failure in 720p WebM transcode
Closed, ResolvedPublic

Description

Transcoding a local copy of the 1920x1088 test file from http://prototype.wikimedia.org/tmh/images/b/b2/File-Arborophila_brunneopectus_pair_feeding_-_Kaeng_Krachan.ogv

Ubuntu 11.04 x86_64 system with ffmpeg stock 0.6.2-4:0.6.2-1ubuntu1.

The '480p' WebM file renders fine, but the '720p' WebM version fails. The .queue.webm file got written out to about 37 of the 42 seconds of input data, and is missing index etc.

The recorded 'error' looks like regular output:

  • 5. row ******* transcode_id: 20 transcode_image_name: File-Arborophila_brunneopectus_pair_feeding_-_Kaeng_Krachan.ogv transcode_key: 480_900kbs.webm transcode_error: transcode_time_addjob: 2011-05-28 00:03:06

transcode_time_startwork: 2011-05-28 00:09:05

transcode_time_success: 2011-05-28 00:10:48
  transcode_time_error: NULL

transcode_final_bitrate: 888048

  • 6. row ******* transcode_id: 21 transcode_image_name: File-Arborophila_brunneopectus_pair_feeding_-_Kaeng_Krachan.ogv transcode_key: 720_VBR.webm transcode_error: FFmpeg version 0.6.2-4:0.6.2-1ubuntu1, Copyright (c) 2000-2010 the Libav developers built on Mar 22 2011 15:55:04 with gcc 4.5.2 configuration: --extra-version=4:0.6.2-1ubuntu1 --prefix=/usr --enable-avfilter --enable-avfilter-lavf --enable-vdpau --enable-bzlib --enable-libgsm --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-pthreads --enable-zlib --enable-libvpx --disable-stripping --enable-runtime-cpudetect --enable-vaapi --enable-gpl --enable-postproc --enable-swscale --enable-x11grab --enable-libdc1394 --enable-shared --disable-static libavutil 50.15. 1 / 50.15. 1 libavcodec 52.72. 2 / 52.72. 2 libavformat 52.64. 2 / 52.64. 2 libavdevice 52. 2. 0 / 52. 2. 0 libavfilter 1.19. 0 / 1.19. 0 libswscale 0.11. 0 / 0.11. 0 libpostproc 51. 2. 0 / 51. 2. 0

Input #0, ogg, from '/var/www/trunk/images/b/b2/File-Arborophila_brunneopectus_pair_feeding_-_Kaeng_Krachan.ogv':

Duration: 00:00:42.20, start: 0.000000, bitrate: 12632 kb/s
  Stream #0.0: Data: skeleton
  Stream #0.1: Video: theora, yuv420p, 1920x1088, 25 fps, 25 tbr, 25 tbn, 25 tbc
  Stream #0.2: Audio: vorbis, 48000 Hz, stereo, s16, 192 kb/s
  Metadata:
    ENCODER         : ffmpeg2theora-0.26+svn16924
    SOURCE_OSHASH   : cd67a9923275c17c

[libvpx @ 0x15ae000]v0.9.6
Output #0, webm, to '/var/www/trunk/images/thumb/b/b2/File-Arborophila_brunneopectus_pair_feeding_-_Kaeng_Krachan.ogv/File-Arborophila_brunneopectus_pair_feeding_-_Kaeng_Krachan.ogv.720_VBR.webm.queue.webm':

Metadata:
  encoder         : Lavf52.64.2
  Stream #0.0: Video: libvpx, yuv420p, 1280x725, q=19-19, 200 kb/s, 1k tbn, 25 tbc
  Stream #0.1: Audio: libvorbis, 48000 Hz, stereo, s16, 64 kb/s
  Metadata:
    ENCODER         : ffmpeg2theora-0.26+svn16924
    SOURCE_OSHASH   : cd67a9923275c17c

Stream mapping:

Stream #0.1 -> #0.0
Stream #0.2 -> #0.1

Press [q] to stop encoding
frame= 945 fps= 10 q=0.0 size= 22121kB time=37.59 bitrate=4821.1kbits/s

transcode_time_addjob: 2011-05-28 00:03:06

transcode_time_startwork: 2011-05-28 00:07:31

transcode_time_success: NULL
  transcode_time_error: 2011-05-28 00:09:05

transcode_final_bitrate: 0

Using these shell memory settings to override the defaults:

$wgMaxShellMemory = 1024000;
$wgMaxShellFileSize = 1024000;

Possibly these are not enough and ffmpeg is dying; README recommends 102400*64 for $wgMaxShellMemory, though there are no recommendations on the wiki page.

Is there any way we can detect a failed process specifically?


Version: unspecified
Severity: normal

Details

Reference
bz29184

Event Timeline

bzimport raised the priority of this task from to Needs Triage.Nov 21 2014, 11:33 PM
bzimport set Reference to bz29184.

mdale wrote:

it does look like a good encode from the error... It does have a high bitrate ... Maybe we need to recommend more memory for ffmpeg?

Can you upload or put this asset somewhere so Jan and I can take a look at it?

Jan is there a good target limit for memory that ffmpeg can consume when dealing with normal range of HD content?

The 64Megs recommendation is not based in science of any kind.

mdale wrote:

I have added a new features help with this:

  1. We list transcode status and error on the image page, that would show why the transcode failed.
  2. We use the --maxtime argument when calling the runJobs.php script and with r90068 we enforce a maxtime for transcodes.
  3. In r90068 I have made transcode calls run in the background and added a check for growing file function that checks if the target file is growing. If the target file stops growing we kill the process and flag the transcode as an error. This should make it a lot harder for frozen transcodes to stay around for any substantial amount of time.
Gilles triaged this task as Unbreak Now! priority.Dec 4 2014, 10:10 AM
Gilles moved this task from Untriaged to Done on the Multimedia board.
Gilles lowered the priority of this task from Unbreak Now! to Needs Triage.Dec 4 2014, 11:23 AM