Page MenuHomePhabricator

TimedMediaHandler passes unrecognized option `-fpsmax` leading to transcoding failures
Closed, ResolvedPublicBUG REPORT

Description

Summary

TimedMediaHandler change #803308 passes the -fpsmax option to FFmpeg. The option has been released with FFmpeg 4.4 but the Wikimedia infrastructure has 4.1.9-0+deb10u1 which leads to:

Unrecognized option 'fpsmax'. Error splitting the argument list: Option not found

It seems to happen when TimedMediaHandler is unable to retrieve the framerate from the source file, it would then passes -fpsmax to enforce a maximum framerate.


Steps to replicate the issue (include links if applicable):

  • Record a video on a Canon R5 4K-D at 50.00fps
  • Transcode to webm/VP9/Vorbis using handbreak "slower" encoder preset. Same result stripping out sound entirely
  • Upload to commons using [[Special:upload]]

What happens?:

The various transcodes to lower quality/size throw up a massive error and don't happen.

Examples (note other than horse plowing these are loud):

*https://commons.wikimedia.org/wiki/File:Fv101_Scorpion_Gdsf_2022_56.webm
*https://commons.wikimedia.org/wiki/File:Horse_Plowing_Gdsf_2022.webm

Seems to be ok with 4K 25FPS

*https://commons.wikimedia.org/wiki/File:Jacobsen_garden_tractor_1963.webm

What should have happened instead?:

Videos should have successfully transcoded to lower size

Other information (browser name/version, screenshots, etc.):

The error message states: Unrecognized option 'fpsmax'. Error splitting the argument list: Option not found

Full error with newlines inserted for readability purposes:

full error
/usr/bin/ffmpeg \
  -nostdin -y -i '/tmp/localcopy_1a29613db174.webm' \
  -fpsmax 60 \
  -threads 8 -row-mt 1 -pix_fmt yuv420p -vcodec libvpx-vp9 -tile-columns '3' -auto-alt-ref 0 -lag-in-frames '0' \
  -quality good -speed 4 -f webm -g '240' \
  -b:v 17200000 -s 3840x2026 -max_muxing_queue_size 1024 -an \
   -pass '1' -passlogfile '/tmp/transcode_2160p.vp9.webm3c3f4f63db69.webm.log' /dev/null

Exitcode: 1
Memory: 4194304
ffmpeg version 4.1.9-0+deb10u1
Copyright (c) 2000-2022 the FFmpeg developers built with gcc 8 (Debian 8.3.0-6) configuration:
--prefix=/usr --extra-version=0+deb10u1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared

libavutil 56. 22.100 / 56. 22.100
libavcodec 58. 35.100 / 58. 35.100
libavformat 58. 20.100 / 58. 20.100
libavdevice 58. 5.100 / 58. 5.100
libavfilter 7. 40.101 / 7. 40.101 
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 3.100 / 5. 3.100
libswresample 3. 3.100 / 3. 3.100 libpostproc 55. 3.100 / 55. 3.100

Unrecognized option 'fpsmax'. Error splitting the argument list: Option not found

Event Timeline

Happened to me today, so I took a look at the code. This is apparently breaking transcode of all files.

I'd appear that this is caused by https://gerrit.wikimedia.org/r/c/mediawiki/extensions/TimedMediaHandler/+/803308 by @brion.

According to @MoritzMuehlenhoff on #wikimedia-operations:

<moritzm> hauskater: -fpsmax was introduced in ffmpeg 4.4 via https://git.ffmpeg.org/gitweb/ffmpeg.git/commit/d99cc1782563672bcdb46fb5ec51135847db8c99, but the video scalers use 4.1.9

So the solution would either remove -fpsmax from the TMH code for now or try to upgrade our ffmpeg to the latest stable version?

TheDJ triaged this task as High priority.Sep 8 2022, 7:25 AM
TheDJ added a project: Train Deployments.

This is from last weeks train, but definitely seems like something that should be rolled back or fixed asap. Tagging T314189: 1.39.0-wmf.28 deployment blockers

TheDJ renamed this task from The transcoding that downscales videos appears to error out on 4K 50FPS footage to The transcoding of video files fails.Sep 8 2022, 7:26 AM
hashar raised the priority of this task from High to Unbreak Now!.Sep 8 2022, 7:51 AM
hashar subscribed.

The transcoding is done by the MediaWiki extensions TimedMediaHandler, looking at the code history, there is a single commit matching fpsmax: 21a48ef249a0afcbff44aa44794cd226436d8304 / https://gerrit.wikimedia.org/r/c/mediawiki/extensions/TimedMediaHandler/+/803308 for T312152

That got merged on August 24th and is in the branch wmf/1.39.0-wmf.27. We did the deployment to commons last week, most probably on Wednesday August 31st if the usual schedule has been respected (we might have delayed the train and landed that version on commons on Sept 1st).

Based on ffmpeg source code to process the command line options, -fpsmax got added by https://github.com/FFmpeg/FFmpeg/commit/d99cc1782563672bcdb46fb5ec51135847db8c99 which is in tags:

n5.2-dev
n5.1.1 n5.1
n5.1-dev
n5.0.1
n5.0
n4.5-dev
n4.4.2
n4.4.1
n4.4

Our version of ffmpeg is 4.1.9-0+deb10u1 and thus does not have the -fpsmax option.

hashar renamed this task from The transcoding of video files fails to TimedMediaHandler passes unrecognized option `-fpsmax` leading to transcoding failures.Sep 8 2022, 7:56 AM
hashar updated the task description. (Show Details)

Change 834530 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/extensions/TimedMediaHandler@master] Revert "Video transcode config cleanup"

https://gerrit.wikimedia.org/r/834530

Change 834530 merged by jenkins-bot:

[mediawiki/extensions/TimedMediaHandler@master] Revert "Video transcode config cleanup"

https://gerrit.wikimedia.org/r/834530

Since the changes were merged into the master branch, may Wikimedia Commons profit from it now, or does it have to wait until the commit comes with a new release?

Change 836997 had a related patch set uploaded (by Martineznovo; author: Jforrester):

[mediawiki/extensions/TimedMediaHandler@REL1_39] Revert "Video transcode config cleanup"

https://gerrit.wikimedia.org/r/836997

Since the changes were merged into the master branch, may Wikimedia Commons profit from it now, or does it have to wait until the commit comes with a new release?

It will be deployed in the weekly train, which will arrive on Commons on Wednesday.

Raphoraph changed the task status from Open to In Progress.Oct 11 2022, 1:41 AM
Raphoraph lowered the priority of this task from Unbreak Now! to Medium.

Can confirm it now works. However, the transcoding must be manually reinitialized on failed files.

Change 836997 merged by jenkins-bot:

[mediawiki/extensions/TimedMediaHandler@REL1_39] Revert "Video transcode config cleanup"

https://gerrit.wikimedia.org/r/836997

TheDJ claimed this task.
TheDJ reassigned this task from TheDJ to Jdforrester-WMF.