Note that this should eliminate or reduce the encoding-time penalty on VP9 by letting us use more threads per conversion. This is particularly important for very long videos at high resolution (conference talks and such), where otherwise we might run past the time limits on the total encoding.

In Debian testing & unstable, this'll be the libvpx5 package and updated ffmpeg & libavcodec57. They do not yet appear in stretch-backports...?

To test if ffmpeg version is built & linked against the correct library:

ffmpeg -row-mt 1

will return either a general error message if supported, or Unrecognized option 'row-mt'. if not.

@brion I'll look into a stretch backport on Friday.

Awesome thanks! I _think_ it should be straightforward. :)

I started to look into that; the ffmpeg version in Debian stable (3.2) doesn't yet support row-mt. I'll see whether I can sanely backport that to 3.2 (otherwise we'd have to follow the various ffmpeg releases in testing to catch up with security updates (currently 3.4, soon 4.0), which would still be kinda manageable, but I'd still prefer to stick with 3.2 if possible to be able to simply rebuild the Debian updates.

Thanks! Unfortunately it looks like 3.2 doesn't include support for the option. We'll either need to update to 3.3 or 3.4 or add in a backport of https://github.com/FFmpeg/FFmpeg/commit/734d760e2fb2621040edef3536b5935e7bc45351 which might be easier.

I created a libvpx 1.7 backport, backported the patch to support mt-row to 3.2 (so that we can stick closely to the 3.2 packages shipped in stretch-security) and rebuilt ffmpeg. That worked fine, I'll rebuild this via the our build system with proper versioning etc. next week.

jmm@hullmann:~/ffmpeg$ cat /etc/issue
Debian GNU/Linux 9 \n \l

jmm@hullmann:~/ffmpeg$ ffmpeg -row-mt 1
ffmpeg version 3.2.10-1~deb9u1 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
  configuration: --prefix=/usr --extra-version='1~deb9u1' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libebur128 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  libavutil      55. 34.101 / 55. 34.101
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.101 / 57. 56.101
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
Trailing options were found on the commandline.
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'

Change 443052 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Add component/vp9

https://gerrit.wikimedia.org/r/443052

Change 443052 merged by Muehlenhoff:
[operations/puppet@production] Add component/vp9

https://gerrit.wikimedia.org/r/443052

Mentioned in SAL (#wikimedia-operations) [2018-06-29T09:18:34Z] <moritzm> uploaded libvpx 1.7.0-3+wmf1 to apt.wikimedia.org/stretch-wikimedia (component/vp9) (T190333)

Mentioned in SAL (#wikimedia-operations) [2018-06-29T12:30:55Z] <moritzm> uploaded ffmpeg 3.2.10-1~deb9u1+wmf1 to apt.wikimedia.org/stretch-wikimedia (component/vp9) (linked against libvpx 1.7 and with backported row-mt support) (T190333)

I've backported the row-mt patch to ffmpeg 3.2. This allows us to stick with the ffmpeg security updates for Debian stretch (we'll still need to rebuild them).

There's now a new repository component "component/vp9" which includes a backport of libvpx 1.7 and ffmpeg compiled with the patch above and linked against 1.7. If anyone wants to test this, use the following apt source:

deb http://apt.wikimedia.org/wikimedia stretch-wikimedia component/vp9
deb-src http://apt.wikimedia.org/wikimedia stretch-wikimedia component/vp9

I've run a number of tests and it all seems to work fine. To measure the performance gain I re-encoded https://commons.wikimedia.org/wiki/File:Wall_of_Death_-_Pitts_Todeswand_2017_-_Jagath_Perera.webm from VP8 to VP9 one mw2446 (one of the codfw video scalers) (which reduces the video file size from 48M to 31M):

Encoding duration using a single CPU core:

time ffmpeg -i Wall_of_Death_-_Pitts_Todeswand_2017_-_Jagath_Perera.webm -c:v libvpx-vp9 -b:v 5M reencode.webm
real    14m36.313s
user    15m2.868s
sys     0m0.280s

Encoding duration using eight CPU cores:

time ffmpeg -i Wall_of_Death_-_Pitts_Todeswand_2017_-_Jagath_Perera.webm -c:v libvpx-vp9 -threads 8 -b:v 5M reencode-3.webm
real    5m31.011s
user    19m6.652s
sys     0m3.792s

Encoding duration using eight CPU cores and row-mt:

time ffmpeg -i Wall_of_Death_-_Pitts_Todeswand_2017_-_Jagath_Perera.webm -c:v libvpx-vp9 -threads 8 -row-mt 1 -b:v 5M reencode-2.webm
real    3m36.301s
user    22m28.812s
sys     0m4.020s

@brion: This isn't rolled out yet (not suitable for a Friday), I can do that next week.

Change 443369 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Add component/vp9 to video scalers

https://gerrit.wikimedia.org/r/443369

Change 443369 merged by Muehlenhoff:
[operations/puppet@production] Add component/vp9 to video scalers

https://gerrit.wikimedia.org/r/443369

Mentioned in SAL (#wikimedia-operations) [2018-07-03T07:18:09Z] <moritzm> upgrading ffmpeg on mw1307 (T190333)

Mentioned in SAL (#wikimedia-operations) [2018-07-06T07:12:20Z] <moritzm> upgrading remaining video scalers to vp9-row-mt enabled ffmpeg build (T190333)