Page MenuHomePhabricator

Update ffmpeg2theora package to fix additional ogv transcode failures
Closed, ResolvedPublic

Description

most ogv transcodes are failing on https://commons.wikimedia.org/wiki/File:The_President_Delivers_a_Statement_on_the_Shooting_in_Oregon.webm -- check whether recent re-enabling of 2-pass encoding is breaking again or something else is wrong

Event Timeline

brion claimed this task.
brion raised the priority of this task from to Needs Triage.
brion updated the task description. (Show Details)
brion added a project: TimedMediaHandler.
brion subscribed.

Ok it's not the 2-pass -- that doesn't seem to have landed yet and it's not using them in the segfaulting errs.

We appear not to have debug symbols for ffmpeg2theora, so this backtrace I got from a manual run on mw1152 is not super useful:

[swscaler @ 0x66d040] Warning: data is not aligned! This can lead to a speedloss
  0:12:43.39 audio: 55kbps video: 466kbps, ET: 00:00:00, est. size: 47.5 MB   
Program received signal SIGSEGV, Segmentation fault.
0x0000000000412af7 in ?? ()
(gdb) bt
#0  0x0000000000412af7 in ?? ()
#1  0x0000000000409fb8 in ?? ()
#2  0x000000000040d30f in ?? ()
#3  0x00007ffff437dec5 in __libc_start_main (main=0x40b297, argc=24, argv=0x7fffffffea38, 
    init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffea28)
    at libc-start.c:287
#4  0x0000000000404419 in ?? ()
(gdb)

*headdesk*

Managed to build with debug symbols:

[swscaler @ 0x6698c0] Warning: data is not aligned! This can lead to a speedloss
  0:12:43.39 audio: 55kbps video: 466kbps, ET: 00:00:00, est. size: 47.5 MB   
Program received signal SIGSEGV, Segmentation fault.
oggmux_add_audio (info=info@entry=0x61d120 <info>, buffer=buffer@entry=0x700f20, samples=samples@entry=1024, 
    e_o_s=e_o_s@entry=1) at src/theorautils.c:1254
1254	                vorbis_buffer[k][i] = ((const float  *)buffer[j])[i];
(gdb) bt
#0  oggmux_add_audio (info=info@entry=0x61d120 <info>, buffer=buffer@entry=0x700f20, samples=samples@entry=1024, 
    e_o_s=e_o_s@entry=1) at src/theorautils.c:1254
#1  0x00000000004080d9 in ff2theora_output (this=this@entry=0x620a30) at src/ffmpeg2theora.c:1619
#2  0x0000000000405202 in main (argc=<optimized out>, argv=<optimized out>) at src/ffmpeg2theora.c:3028
(gdb)

It seems that in the very last packet it's crapping out; either the buffer is bogus or it's been overwritten with something, making it bogus.

So it looks like what's happening is the audio track is slightly shorter than the video track (?) and at the end of the file we don't have an audio frame, but it's mistakenly trying to process one with a no-longer-valid buffer and explodes.

Updated patch which seems to work -- will test more thoroughly when not on an airplane: https://github.com/brion/ffmpeg2theora/commit/70905befe8c80604f7312c8aeffbdcfb49fe128d

brion added a project: acl*sre-team.
brion added a subscriber: Paladox.

Adding operations (is that right?) -- need to update ffmpeg2theora package in our apt repo with updated patch.

brion renamed this task from Investigate additional ogv transcode failures to Update ffmpeg2theora package to fix additional ogv transcode failures.Oct 3 2015, 9:17 PM
brion set Security to None.

(Note last major update to our ffmpeg2theora package was for T69953 -- just need to swap out the patch file for newer one.)

Updated patch:

drop into the debian/patches/ replacing the old fix-resample-sefault.patch.

ok so I've updated the patch and uploaded 0.29.0~git+20150813-2, also the package has now a corresponding operations/debs/ffmpeg2theora repository that can be updated.

When will this be deployed on production since commons still shows the error.

Could it be to do with samplerate sinve 720p and 1080p for ogv doint do that instead they do videoQuality and audioQuality and same for vp8.

It seems 480p is different to 720p

480p

			array(
				'maxSize'                    => '854x480',
				'videoBitrate'               => '2048',
				'audioQuality'               => '2',
				'samplerate'                 => '44100',
				'channels'                   => '2',
				'noUpscaling'                => 'true',
				'twopass'                    => 'true',
				'optimize'                   => 'true',
				'keyframeInterval'           => '128',
				'bufDelay'                   => '256',
				'videoCodec'                 => 'theora',
				'type'                       => 'video/ogg; codecs="theora, vorbis"',
			),

720p

		WebVideoTranscode::ENC_OGV_720P =>
			array(
				'maxSize'                    => '1280x720',
				'videoQuality'               => 6,
				'audioQuality'               => 3,
				'noUpscaling'                => 'true',
				'twopass'                    => 'true',
				'optimize'                   => 'true',
				'keyframeInterval'           => '128',
				'videoCodec'                 => 'theora',
				'type'                       => 'video/ogg; codecs="theora, vorbis"',
			),

Seems that the video is a vp9 video using vorbus not opus.

@fgiunchedi woohoo looks good! Is there any way to force the video scalers to install the update? It doesn't seem to have gone out to them yet. Thanks!

Fixed problem.

Its to do with that video was using vorbis. Currently our setting in timedmediahandler only supports using opus with vp9.

Please see this test video

that fixes the problem.

No @Paladox, input files can use whatever combination they like, that doesn't matter at all here. That particular file was a particular size that exposed a particular bug in ffmpeg2theora, which is what this this patch fixes.

@brion looks good, videoscalers are running the latest version

salt --out=raw -b 1 -t 300 -v -C 'G@cluster:videoscaler' cmd.run 'apt-get install -y ffmpeg2theora'

Oh ok. I updated the file anyways with lower memory usage and corrected audio format.

@Paladox PLEASE STOP interfering with bug testing. Leave the files alone.

@brion could we use the version I uploaded which used less memory and used correct audio format.

@Paladox please just leave it alone. Stop messing with the file. Stop messing with this bug.

Ok sorry. I wasent aware at the time that you were testing it only that you were using it as an example showing there was a bug.

Ok, the 360p/240p/160p ogvs rendered correctly! Yay!

The 480p ogg and the webms were damaged

Have to wait another 48 minutes to re-run and fix the webm & 480p ogv transcodes. :(

Ok it all seems to have worked out. :) Thanks for the package update @fgiunchedi!