Theora video thumbnails corrupt
Closed, ResolvedPublic

Description

Author: camjsb7j9g

Description:
Test_20100411_Vdubtestrgbcube4secs_ifps.ogv with empty thumbnail

Between 03:23, 8 April 2010 and 08:40, 9 April 2010 new video uploads started to render with empty or corrupt thumbnails (Firefox, Google Chrome, Opera and Safari, with cache and file purging; thumbtime also tested). New uploads are still rendering incorrectly. I *believe* I have eliminated file corruption because the file uploaded is identical to the file on my PC (I download the file from mediawiki and use emacs to compare).

See http://commons.wikimedia.org/wiki/Commons:Village_pump#Video_thumbnails_corrupt_since_9_April

Duplicated problem on skins: Monobook, Vector and Classic. Other skins not tried.

How to replicate:

1: choose any small video from Category:Calibration videos at http://commons.wikimedia.org/wiki/Category:Calibration_videos and download to computer

2: upload same file with name prefixed with "Test_reupload_"; click the "ignore warning and save anyway" button

3: see the corrupt or empty video thumbnail

Example attached from: http://commons.wikimedia.org/wiki/File:Test_20100411_Vdubtestrgbcube4secs_ifps12fr12_q10.ogv


Version: unspecified
Severity: major
OS: Windows Vista
Platform: PC

Attached:

bzimport added a subscriber: Unknown Object (MLST).
bzimport set Reference to bz23160.
bzimport created this task.Via LegacyApr 11 2010, 10:39 PM
TheDJ added a comment.Via ConduitApr 11 2010, 11:26 PM

The empty thumbs are due to the fact that there are no key frames following the seek point (midway file). We should probably always start at frame 0 when a movie is under 10seconds.

The other issue is of a similar nature and also related to keyframes.

What triggered this behavior, i'm not sure of, but most likely ffmpeg was upgraded on the scalers. I have asked Roan, and the servers currently run r11872+debian_3:0.svn20080206-12ubuntu

If someone could run the following on a scaling server and extract the debug info from ffmpeg please. That might help:

ffmpeg -ss 9.5 -i /mnt/upload6/wikipedia/commons/b/bd/Translational_motion_gif.ogv -f mjpeg -an -vframes 1 ./test.jpg

I made an educated guess at the path. Might not be 100% correct.

TheDJ added a comment.Via ConduitApr 11 2010, 11:42 PM

I note that wmf-deployment has some OggHandler/OggHandler_body.php changes from mdale, that are not currently in wmf/1.16wmf4

TheDJ added a comment.Via ConduitApr 12 2010, 12:09 AM
  • Bug 23163 has been marked as a duplicate of this bug. ***
Ragesoss added a comment.Via ConduitApr 12 2010, 5:46 PM

Is there a workaround, or are all thumbnails on new videos broken for now?

bzimport added a comment.Via ConduitApr 12 2010, 9:42 PM

gmaxwell wrote:

The thumbnails for the files linked in in Bug 23163 don't really look like failing to decode starting at a keyframe. They look like the result of decoding with the broken theora encoder which was included in ffmpeg up until about a year ago.

At least of the cluster was upgraded to fix this at some point, but perhaps some old software is still out there on some systems.

TheDJ added a comment.Via ConduitApr 12 2010, 11:34 PM

(In reply to comment #4)

Is there a workaround, or are all thumbnails on new videos broken for now?

there is currently no workaround. When everyone has landed in Berlin, Tim Starling and/or Michael Dale will have to look at it probably.

bzimport added a comment.Via ConduitApr 12 2010, 11:46 PM

gmaxwell wrote:

(In reply to comment #5)

with the broken theora encoder which was included in ffmpeg up until about a

*decoder* ugh.

http://lists.wikimedia.org/pipermail/wikitech-l/2010-February/046706.html < this thread is likely relevant too.

TheDJ added a comment.Via ConduitApr 12 2010, 11:51 PM

(In reply to comment #7)

http://lists.wikimedia.org/pipermail/wikitech-l/2010-February/046706.html <
this thread is likely relevant too.

Yes, that's exactly why I pointed out the differences between wmf-deployment and 1.16wmf4

Catrope added a comment.Via ConduitApr 14 2010, 8:51 AM

Created attachment 7290
Debug output

Debug output as requested in comment #1.

Attached: debug.log

bzimport added a comment.Via ConduitApr 14 2010, 12:51 PM

gmaxwell wrote:

The ffmpeg in that debugging output is too old. Even absent the seeking problems it will not reliably produce correct output.

bzimport added a comment.Via ConduitApr 15 2010, 4:29 PM

mdale wrote:

there is also this code review comment from Tim that says that oggThumb is in fact oky?
http://www.mediawiki.org/wiki/Special:Code/MediaWiki/62223#c5744

... but would have to go into the "code review queue" ? But the fix was already in the deployment branch without any suggested changes. So Tim forgot to merge it before he deployed it ?

I can send a patch to make wmf4 what was once wmf-deployment adding in Tim's fix to bug 22388 ? Does that sound reasonable?

And we the other parts of oggHandler can go back into trunk for review ? ( i.e transcoding so have some basic consistency in the bitrates of embed videos ? )

bzimport added a comment.Via ConduitApr 17 2010, 12:28 AM

mdale wrote:

Restored the previously deployed ( working ) oggThumb code to oggHandler trunk. r65149

Its kind of tricky to test since you need the svn version of oggThumb ( that has already been deployed )

TheDJ added a comment.Via ConduitApr 18 2010, 3:26 PM
  • Bug 22100 has been marked as a duplicate of this bug. ***
bzimport added a comment.Via ConduitApr 21 2010, 10:26 PM

mdale wrote:

any update on this?

Almost all newly uploaded videos continue to have broken thumbnails ...

Should I be giving opts people instruction to compile ffmpeg to deploy less broken thumbnailing?
Or can we put the code that was working for a few months back into operation while this gets sorted?

bzimport added a comment.Via ConduitApr 21 2010, 10:36 PM

gmaxwell wrote:

FWIW— Yet another Ffmpeg-violates-the-spec causing failed decoding problem was discovered and fixed today. This isn't a case that people should have been hitting with any frequency but it's a reason to use only the most current development ffmpeg if ffmpeg is continued to be used.

bzimport added a comment.Via ConduitApr 28 2010, 8:03 PM

camjsb7j9g wrote:

FWIW2- ffmpeg2theora version 0.24 is giving good thumbnails for me lately. Versions 0.25, 0.26 and from 2010-03-16 always gives garbled or missing thumbnails. For small examples see [http://commons.wikimedia.org/wiki/File:SDO 10 second youtube 1080 pixels DA8C8E04d01.flv 320x240 br270.ogv] and [http://commons.wikimedia.org/wiki/File:Solar Dynamics Observatory xo with ffmpeg2theora-0.24 320x240 br270.ogv]: scroll down to "Source" for the exact commands used to encode. I also posted this at [http://commons.wikimedia.org/wiki/Help_talk:Converting_video#Video_thumbnail_error] and [http://commons.wikimedia.org/wiki/Commons:Village_pump#Theora_thumbnails].

bzimport added a comment.Via ConduitApr 28 2010, 8:07 PM

camjsb7j9g wrote:

(In reply to comment #16)

FWIW2- ffmpeg2theora version 0.24 is giving good thumbnails for me lately.
Versions 0.25, 0.26 and from 2010-03-16 always gives garbled or missing
thumbnails. For small examples see [http://commons.wikimedia.org/wiki/File:SDO
10 second youtube 1080 pixels DA8C8E04d01.flv 320x240 br270.ogv] and
[http://commons.wikimedia.org/wiki/File:Solar Dynamics Observatory xo with
ffmpeg2theora-0.24 320x240 br270.ogv]: scroll down to "Source" for the exact
commands used to encode. I also posted this at
[http://commons.wikimedia.org/wiki/Help_talk:Converting_video#Video_thumbnail_error]
and [http://commons.wikimedia.org/wiki/Commons:Village_pump#Theora_thumbnails].

The above URLs should read:

http://commons.wikimedia.org/wiki/File:SDO_10_second_youtube_1080_pixels_DA8C8E04d01.flv_320x240_br270.ogv

and

http://commons.wikimedia.org/wiki/File:Solar_Dynamics_Observatory_xo_with_ffmpeg2theora-0.24_320x240_br270.ogv

also

http://commons.wikimedia.org/wiki/File:Helioseismic_and_Magnetic_Imager_First_Light_Briefing_320x180_ffmpeg2theora-0.24.ogv

tstarling added a comment.Via ConduitApr 30 2010, 12:43 AM

I've rewritten the oggThumb support in OggHandler and I've made some substantial changes to oggThumb itself in a development branch on sf.net. I'm going to try handing this over to Mark Bergsma for packaging.

tstarling added a comment.Via ConduitMay 4 2010, 1:34 AM

Fixed now. Any remaining cases should be fixable by doing action=purge on the image description page followed by forced refresh (e.g. shift-reload in Firefox).

Add Comment