Page MenuHomePhabricator

ogv transcode error
Closed, ResolvedPublic

Description

Hi when I click reset transcode at https://commons.wikimedia.org/wiki/File:4K_resolution_sample.ogv

It is bring up an error

Error says

'/usr/bin/ffmpeg2theora' '/tmp/localcopy_edda79afa069-1.ogv' -V '2048' -a '2' -H '44100' -c '2' --no-upscaling --two-pass --optimize --keyint '128' --buf-delay '256' --width '854' --height '480' --aspect '854:480' -o '/tmp/transcode_480p.ogv417ac20e08b2-1.ogv'

Exitcode: 1
Memory: 4194304

[ogg @ 0x187d700] Unknown skeleton version 4.0
[ogg @ 0x187d700] max_analyze_duration reached
Input #0, ogg, from '/tmp/localcopy_edda79afa069-1.ogv':

Duration: 00:00:44.00, start: 0.000000, bitrate: 1483 kb/s
  Stream #0.0: Data: skeleton
  Stream #0.1: Video: theora, yuv420p, 4096x2304, 6 fps, 6 tbr, 6 tbn, 6 tbc
  Stream #0.2: Audio: vorbis, 44100 Hz, stereo, s16, 64 kb/s
  Metadata:
    ENCODER         : ffmpeg2theora-0.27
    SOURCE_OSHASH   : 545c80ef13f47ed2
Pixel Aspect Ratio: 1.00/1   Frame Aspect Ratio: 1.78/1
Resize: 4096x2304 => 854x480

Scanning first pass pos: 0:00:00.00 ET: 00:00:00
Scanning first pass pos: 0:00:00.66 ET: 00:00:00
Scanning first pass pos: 0:00:01.16 ET: 00:00:36
Scanning first pass pos: 0:00:01.83 ET: 00:00:23
Scanning first pass pos: 0:00:02.33 ET: 00:00:17
Scanning first pass pos: 0:00:03.00 ET: 00:00:27
Scanning first pass pos: 0:00:03.66 ET: 00:00:22
Scanning first pass pos: 0:00:04.16 ET: 00:00:19
Scanning first pass pos: 0:00:04.83 ET: 00:00:24
Scanning first pass pos: 0:00:05.50 ET: 00:00:21
Scanning first pass pos: 0:00:06.16 ET: 00:00:18
Scanning first pass pos: 0:00:06.83 ET: 00:00:21
Scanning first pass pos: 0:00:07.50 ET: 00:00:19
Scanning first pass pos: 0:00:08.16 ET: 00:00:17
Scanning first pass pos: 0:00:08.83 ET: 00:00:15
Scanning first pass pos: 0:00:09.50 ET: 00:00:18
Scanning first pass pos: 0:00:10.16 ET: 00:00:16
Scanning first pass pos: 0:00:10.83 ET: 00:00:15
Scanning first pass pos: 0:00:11.50 ET: 00:00:16
Scanning first pass pos: 0:00:12.16 ET: 00:00:15
Scanning first pass pos: 0:00:12.83 ET: 00:00:14
Scanning first pass pos: 0:00:13.50 ET: 00:00:15
Scanning first pass pos: 0:00:14.16 ET: 00:00:14
Scanning first pass pos: 0:00:14.83 ET: 00:00:13
Scanning first pass pos: 0:00:15.50 ET: 00:00:14
Scanning first pass pos: 0:00:16.16 ET: 00:00:13
Scanning first pass pos: 0:00:16.83 ET: 00:00:14
Scanning first pass pos: 0:00:17.50 ET: 00:00:13
Scanning first pass pos: 0:00:18.16 ET: 00:00:12
Scanning first pass pos: 0:00:18.83 ET: 00:00:13
Scanning first pass pos: 0:00:19.50 ET: 00:00:12
Scanning first pass pos: 0:00:20.16 ET: 00:00:13
Scanning first pass pos: 0:00:20.83 ET: 00:00:12
Scanning first pass pos: 0:00:21.50 ET: 00:00:11
Scanning first pass pos: 0:00:22.16 ET: 00:00:10
Scanning first pass pos: 0:00:22.83 ET: 00:00:11
Scanning first pass pos: 0:00:23.50 ET: 00:00:10
Scanning first pass pos: 0:00:24.16 ET: 00:00:09
Scanning first pass pos: 0:00:24.83 ET: 00:00:10
Scanning first pass pos: 0:00:25.50 ET: 00:00:09
Scanning first pass pos: 0:00:26.16 ET: 00:00:08
Scanning first pass pos: 0:00:26.83 ET: 00:00:08
Scanning first pass pos: 0:00:27.50 ET: 00:00:08
Scanning first pass pos: 0:00:28.16 ET: 00:00:07
Scanning first pass pos: 0:00:28.83 ET: 00:00:07
Scanning first pass pos: 0:00:29.50 ET: 00:00:07
Scanning first pass pos: 0:00:30.16 ET: 00:00:06
Scanning first pass pos: 0:00:30.83 ET: 00:00:06
Scanning first pass pos: 0:00:31.50 ET: 00:00:06
Scanning first pass pos: 0:00:32.16 ET: 00:00:05
Scanning first pass pos: 0:00:32.83 ET: 00:00:05
Scanning first pass pos: 0:00:33.50 ET: 00:00:05
Scanning first pass pos: 0:00:34.16 ET: 00:00:04
Scanning first pass pos: 0:00:34.83 ET: 00:00:04
Scanning first pass pos: 0:00:35.50 ET: 00:00:04
Scanning first pass pos: 0:00:36.16 ET: 00:00:04
Scanning first pass pos: 0:00:36.83 ET: 00:00:03
Scanning first pass pos: 0:00:37.50 ET: 00:00:03
Scanning first pass pos: 0:00:38.16 ET: 00:00:03
Scanning first pass pos: 0:00:38.83 ET: 00:00:02
Scanning first pass pos: 0:00:39.50 ET: 00:00:02
Scanning first pass pos: 0:00:40.16 ET: 00:00:02
Scanning first pass pos: 0:00:40.83 ET: 00:00:01
Scanning first pass pos: 0:00:41.50 ET: 00:00:01
Scanning first pass pos: 0:00:42.16 ET: 00:00:00
Scanning first pass pos: 0:00:42.83 ET: 00:00:00
Scanning first pass pos: 0:00:43.50 ET: 00:00:00
Scanning first pass pos: 0:00:44.00 ET: 00:00:00 [ogg @ 0x187d700] Unknown skeleton version 4.0
[ogg @ 0x187d700] max_analyze_duration reached

0:00:00.50 audio: 1kbps video: 6260kbps, ET: 00:33:18, est. size: 10.9 MB   
0:00:01.03 audio: 1kbps video: 2005kbps, ET: 00:15:55, est. size: 6.8 MB   
0:00:01.56 audio: 1kbps video: 2005kbps, ET: 00:10:22, est. size: 4.5 MB   
0:00:02.10 audio: 1kbps video: 669kbps, ET: 00:07:38, est. size: 3.3 MB   
0:00:02.63 audio: 1kbps video: 618kbps, ET: 00:06:16, est. size: 2.7 MB   
0:00:03.17 audio: 1kbps video: 502kbps, ET: 00:05:09, est. size: 2.2 MB   
0:00:03.70 audio: 1kbps video: 365kbps, ET: 00:04:21, est. size: 1.9 MB   
0:00:04.23 audio: 1kbps video: 468kbps, ET: 00:03:45, est. size: 2.4 MB   
0:00:04.77 audio: 1kbps video: 418kbps, ET: 00:03:25, est. size: 2.2 MB   
0:00:05.33 audio: 1kbps video: 366kbps, ET: 00:03:01, est. size: 1.9 MB   
0:00:06.00 audio: 1kbps video: 326kbps, ET: 00:02:38, est. size: 1.7 MB   
0:00:06.66 audio: 1kbps video: 293kbps, ET: 00:02:20, est. size: 1.5 MB   
0:00:07.32 audio: 1kbps video: 286kbps, ET: 00:02:10, est. size: 1.4 MB   
0:00:07.83 audio: 1kbps video: 250kbps, ET: 00:02:00, est. size: 1.3 MB   
0:00:08.39 audio: 1kbps video: 244kbps, ET: 00:01:50, est. size: 1.2 MB   
0:00:08.92 audio: 1kbps video: 266kbps, ET: 00:01:49, est. size: 1.3 MB   
0:00:09.46 audio: 1kbps video: 266kbps, ET: 00:01:42, est. size: 1.3 MB   
0:00:09.99 audio: 1kbps video: 266kbps, ET: 00:01:35, est. size: 1.2 MB   
0:00:10.53 audio: 1kbps video: 266kbps, ET: 00:01:28, est. size: 1.1 MB   
0:00:11.06 audio: 1kbps video: 266kbps, ET: 00:01:23, est. size: 1.1 MB   
0:00:11.59 audio: 1kbps video: 266kbps, ET: 00:01:18, est. size: 1.0 MB   Error submitting pass data in second pass.

Event Timeline

Paladox created this task.Aug 19 2015, 6:30 PM
Paladox raised the priority of this task from to Needs Triage.
Paladox updated the task description. (Show Details)
Paladox added a project: TimedMediaHandler.
Paladox moved this task to Backlog on the TimedMediaHandler board.
Paladox added a subscriber: Paladox.
Restricted Application added a project: Multimedia. · View Herald TranscriptAug 19 2015, 6:30 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Paladox set Security to None.Aug 19 2015, 6:31 PM
Paladox added a subscriber: brion.
Restricted Application added a subscriber: Matanya. · View Herald TranscriptAug 19 2015, 6:31 PM

I think this is high since I have no control off it.

brion claimed this task.Aug 19 2015, 9:12 PM

Checking into this.

brion added a comment.Aug 19 2015, 9:17 PM

Ok I was able to run .ogv updates on another (smaller) file... so it may be specifically related to it being a very large frame size and using more memory/storage resources. Will see if I can replicate the failure reliably.

Ok thanks. It start d recently today since commons was switched to wmf19 branch. It was working earlier today.

brion added a comment.Aug 19 2015, 9:44 PM

Ok, got a similar error on another video (1080p source, no problems rendering it previously):

...
  0:03:33.88 audio: 60kbps video: 509kbps, ET: 00:00:23, est. size: 16.3 MB   
  0:03:34.38 audio: 60kbps video: 509kbps, ET: 00:00:23, est. size: 16.3 MB   
  0:03:34.96 audio: 60kbps video: 510kbps, ET: 00:00:22, est. size: 16.3 MB   Error submitting pass data in second pass.

So something is definitely sometimes failing in the two-pass encoding; on these smallish files it *should* not be encountering memory or file size limits here...

Yes we can trace the problem between the last commit for wmf18 and wmf19 so between them a problem was introduced.

Change 232647 had a related patch set uploaded (by Brion VIBBER):
Re-disable 2-pass Theora encoding temporarily

https://gerrit.wikimedia.org/r/232647

Change 232649 had a related patch set uploaded (by Brion VIBBER):
Re-disable 2-pass Theora encoding temporarily

https://gerrit.wikimedia.org/r/232649

Ok I've scheduled a 'SWAT' update request at https://wikitech.wikimedia.org/wiki/Deployments#Week_of_August_17th for a patch re-disabling the 2-pass encoding on .ogv derivatives.

I'll re-add it after we've updated our transcode servers (which will be running a more up to date ffmpeg2theora, and with a package we can fix if need be!)

Change 232649 merged by jenkins-bot:
Re-disable 2-pass Theora encoding temporarily

https://gerrit.wikimedia.org/r/232649

brion closed this task as Resolved.Aug 19 2015, 11:23 PM

Ok this should be working again, with the two-pass disabled again for now.

Change 232647 merged by jenkins-bot:
Re-disable 2-pass Theora encoding temporarily

https://gerrit.wikimedia.org/r/232647

Hi I am getting this error for webm at https://commons.wikimedia.org/wiki/File:Wikipédia_1.4_fr_1_4k.webm

Error says

'/usr/bin/avconv' -y -i '/tmp/localcopy_559b99165a2c-1.webm' -threads 2 -skip_threshold 0 -bufsize 6000k -rc_init_occupancy 4000 -qmin 1 -qmax 51 -vb '512000' -vcodec libvpx -g '128' -keyint_min '128' -f webm -s 640x360 -an -pass '1' -passlogfile '/tmp/transcode_360p.webma49b1f28f49a-1.webm.log' /dev/null

Exitcode: 1
Memory: 4194304

avconv version 0.8.17-4:0.8.17-0ubuntu0.12.04.1, Copyright (c) 2000-2014 the Libav developers

built on Mar 16 2015 13:26:50 with gcc 4.6.3

[matroska,webm @ 0x22417a0] Unknown/unsupported CodecID V_VP9.
[matroska,webm @ 0x22417a0] max_analyze_duration reached
[matroska,webm @ 0x22417a0] Estimating duration from bitrate, this may be inaccurate
Input #0, matroska,webm, from '/tmp/localcopy_559b99165a2c-1.webm':

Duration: 00:13:43.75, start: 0.000000, bitrate: N/A
  Stream #0.0: Video: [0][0][0][0] / 0x0000, 3840x2160, PAR 1:1 DAR 16:9, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
  Stream #0.1: Audio: vorbis, 44100 Hz, stereo, s16
  Stream #0.2: Audio: vorbis, 44100 Hz, stereo, s16

[buffer @ 0x2351800] Invalid pixel format string '-1'
Error opening filters!

That's unrelated, that's a VP9 file that won't be supported until we upgrade servers. Please don't add unrelated info onto closed bugs.

Jdforrester-WMF triaged this task as High priority.Sep 15 2015, 3:21 PM
Jdforrester-WMF moved this task from Untriaged to Done on the Multimedia board.