Theora video thumbnails corrupt
Closed, ResolvedPublic

Description

Author: camjsb7j9g

Description:
Test_20100411_Vdubtestrgbcube4secs_ifps.ogv with empty thumbnail

Between 03:23, 8 April 2010 and 08:40, 9 April 2010 new video uploads started to render with empty or corrupt thumbnails (Firefox, Google Chrome, Opera and Safari, with cache and file purging; thumbtime also tested). New uploads are still rendering incorrectly. I *believe* I have eliminated file corruption because the file uploaded is identical to the file on my PC (I download the file from mediawiki and use emacs to compare).

See http://commons.wikimedia.org/wiki/Commons:Village_pump#Video_thumbnails_corrupt_since_9_April

Duplicated problem on skins: Monobook, Vector and Classic. Other skins not tried.

How to replicate:

1: choose any small video from Category:Calibration videos at http://commons.wikimedia.org/wiki/Category:Calibration_videos and download to computer

2: upload same file with name prefixed with "Test_reupload_"; click the "ignore warning and save anyway" button

3: see the corrupt or empty video thumbnail

Example attached from: http://commons.wikimedia.org/wiki/File:Test_20100411_Vdubtestrgbcube4secs_ifps12fr12_q10.ogv


Version: unspecified
Severity: major
OS: Windows Vista
Platform: PC

Attached:

bzimport added a subscriber: wikibugs-l.
bzimport set Reference to bz23160.
bzimport created this task.Via LegacyApr 11 2010, 10:39 PM
TheDJ added a comment.Via ConduitApr 11 2010, 11:26 PM

The empty thumbs are due to the fact that there are no key frames following the seek point (midway file). We should probably always start at frame 0 when a movie is under 10seconds.

The other issue is of a similar nature and also related to keyframes.

What triggered this behavior, i'm not sure of, but most likely ffmpeg was upgraded on the scalers. I have asked Roan, and the servers currently run r11872+debian_3:0.svn20080206-12ubuntu

If someone could run the following on a scaling server and extract the debug info from ffmpeg please. That might help:

ffmpeg -ss 9.5 -i /mnt/upload6/wikipedia/commons/b/bd/Translational_motion_gif.ogv -f mjpeg -an -vframes 1 ./test.jpg

I made an educated guess at the path. Might not be 100% correct.

TheDJ added a comment.Via ConduitApr 11 2010, 11:42 PM

I note that wmf-deployment has some OggHandler/OggHandler_body.php changes from mdale, that are not currently in wmf/1.16wmf4

TheDJ added a comment.Via ConduitApr 12 2010, 12:09 AM
  • Bug 23163 has been marked as a duplicate of this bug. ***
Ragesoss added a comment.Via ConduitApr 12 2010, 5:46 PM

Is there a workaround, or are all thumbnails on new videos broken for now?

bzimport added a comment.Via ConduitApr 12 2010, 9:42 PM

gmaxwell wrote:

The thumbnails for the files linked in in Bug 23163 don't really look like failing to decode starting at a keyframe. They look like the result of decoding with the broken theora encoder which was included in ffmpeg up until about a year ago.

At least of the cluster was upgraded to fix this at some point, but perhaps some old software is still out there on some systems.

TheDJ added a comment.Via ConduitApr 12 2010, 11:34 PM

(In reply to comment #4)

Is there a workaround, or are all thumbnails on new videos broken for now?

there is currently no workaround. When everyone has landed in Berlin, Tim Starling and/or Michael Dale will have to look at it probably.

bzimport added a comment.Via ConduitApr 12 2010, 11:46 PM

gmaxwell wrote:

(In reply to comment #5)

with the broken theora encoder which was included in ffmpeg up until about a

*decoder* ugh.

http://lists.wikimedia.org/pipermail/wikitech-l/2010-February/046706.html < this thread is likely relevant too.

TheDJ added a comment.Via ConduitApr 12 2010, 11:51 PM

(In reply to comment #7)

http://lists.wikimedia.org/pipermail/wikitech-l/2010-February/046706.html <
this thread is likely relevant too.

Yes, that's exactly why I pointed out the differences between wmf-deployment and 1.16wmf4

Catrope added a comment.Via ConduitApr 14 2010, 8:51 AM

Created attachment 7290
Debug output

Debug output as requested in comment #1.

Attached: debug.log

bzimport added a comment.Via ConduitApr 14 2010, 12:51 PM

gmaxwell wrote:

The ffmpeg in that debugging output is too old. Even absent the seeking problems it will not reliably produce correct output.

bzimport added a comment.Via ConduitApr 15 2010, 4:29 PM

mdale wrote:

there is also this code review comment from Tim that says that oggThumb is in fact oky?
http://www.mediawiki.org/wiki/Special:Code/MediaWiki/62223#c5744

... but would have to go into the "code review queue" ? But the fix was already in the deployment branch without any suggested changes. So Tim forgot to merge it before he deployed it ?

I can send a patch to make wmf4 what was once wmf-deployment adding in Tim's fix to bug 22388 ? Does that sound reasonable?

And we the other parts of oggHandler can go back into trunk for review ? ( i.e transcoding so have some basic consistency in the bitrates of embed videos ? )

bzimport added a comment.Via ConduitApr 17 2010, 12:28 AM

mdale wrote:

Restored the previously deployed ( working ) oggThumb code to oggHandler trunk. r65149

Its kind of tricky to test since you need the svn version of oggThumb ( that has already been deployed )

TheDJ added a comment.Via ConduitApr 18 2010, 3:26 PM
  • Bug 22100 has been marked as a duplicate of this bug. ***
bzimport added a comment.Via ConduitApr 21 2010, 10:26 PM

mdale wrote:

any update on this?

Almost all newly uploaded videos continue to have broken thumbnails ...

Should I be giving opts people instruction to compile ffmpeg to deploy less broken thumbnailing?
Or can we put the code that was working for a few months back into operation while this gets sorted?

bzimport added a comment.Via ConduitApr 21 2010, 10:36 PM

gmaxwell wrote:

FWIW— Yet another Ffmpeg-violates-the-spec causing failed decoding problem was discovered and fixed today. This isn't a case that people should have been hitting with any frequency but it's a reason to use only the most current development ffmpeg if ffmpeg is continued to be used.

bzimport added a comment.Via ConduitApr 28 2010, 8:03 PM

camjsb7j9g wrote:

FWIW2- ffmpeg2theora version 0.24 is giving good thumbnails for me lately. Versions 0.25, 0.26 and from 2010-03-16 always gives garbled or missing thumbnails. For small examples see [http://commons.wikimedia.org/wiki/File:SDO 10 second youtube 1080 pixels DA8C8E04d01.flv 320x240 br270.ogv] and [http://commons.wikimedia.org/wiki/File:Solar Dynamics Observatory xo with ffmpeg2theora-0.24 320x240 br270.ogv]: scroll down to "Source" for the exact commands used to encode. I also posted this at [http://commons.wikimedia.org/wiki/Help_talk:Converting_video#Video_thumbnail_error] and [http://commons.wikimedia.org/wiki/Commons:Village_pump#Theora_thumbnails].

bzimport added a comment.Via ConduitApr 28 2010, 8:07 PM

camjsb7j9g wrote:

(In reply to comment #16)

FWIW2- ffmpeg2theora version 0.24 is giving good thumbnails for me lately.
Versions 0.25, 0.26 and from 2010-03-16 always gives garbled or missing
thumbnails. For small examples see [http://commons.wikimedia.org/wiki/File:SDO
10 second youtube 1080 pixels DA8C8E04d01.flv 320x240 br270.ogv] and
[http://commons.wikimedia.org/wiki/File:Solar Dynamics Observatory xo with
ffmpeg2theora-0.24 320x240 br270.ogv]: scroll down to "Source" for the exact
commands used to encode. I also posted this at
[http://commons.wikimedia.org/wiki/Help_talk:Converting_video#Video_thumbnail_error]
and [http://commons.wikimedia.org/wiki/Commons:Village_pump#Theora_thumbnails].

The above URLs should read:

http://commons.wikimedia.org/wiki/File:SDO_10_second_youtube_1080_pixels_DA8C8E04d01.flv_320x240_br270.ogv

and

http://commons.wikimedia.org/wiki/File:Solar_Dynamics_Observatory_xo_with_ffmpeg2theora-0.24_320x240_br270.ogv

also

http://commons.wikimedia.org/wiki/File:Helioseismic_and_Magnetic_Imager_First_Light_Briefing_320x180_ffmpeg2theora-0.24.ogv

tstarling added a comment.Via ConduitApr 30 2010, 12:43 AM

I've rewritten the oggThumb support in OggHandler and I've made some substantial changes to oggThumb itself in a development branch on sf.net. I'm going to try handing this over to Mark Bergsma for packaging.

tstarling added a comment.Via ConduitMay 4 2010, 1:34 AM

Fixed now. Any remaining cases should be fixable by doing action=purge on the image description page followed by forced refresh (e.g. shift-reload in Firefox).

Add Comment

Column Prototype
This is a very early prototype of a persistent column. It is not expected to work yet, and leaving it open will activate other new features which will break things. Press "\" (backslash) on your keyboard to close it now.