Page MenuHomePhabricator

Bad bitrate/quality for WebM two-pass encoding with avconv in vagrant
Closed, ResolvedPublic

Description

WebM transcodes created with the configuration in MediaWiki-Vagrant and Labs-Vagrant are much lower quality, and at a lower total bitrate, than expected.

Example... This Ogg transcode is correct bitrate:

http://ogvjs-testing.wmflabs.org/images/transcoded/c/c8/Sneak_Preview_-_Wikipedia_VisualEditor.webm/Sneak_Preview_-_Wikipedia_VisualEditor.webm.360p.ogv

while its WebM sibling is much lower bitrate and visibly much poorer quality:

http://ogvjs-testing.wmflabs.org/images/transcoded/c/c8/Sneak_Preview_-_Wikipedia_VisualEditor.webm/Sneak_Preview_-_Wikipedia_VisualEditor.webm.360p.webm

Disabling the "'twopass' => true" setting in the transcode options in WebVideoTranscode.php fixes it for me, but it seems like this should work with two-pass encoding, especially as I assume that's what's being used in production...

Event Timeline

brion raised the priority of this task from to Needs Triage.
brion updated the task description. (Show Details)
brion added a subscriber: brion.

Note that I can work around this on Ubuntu 15.04 Vivid by switching from avconv to ffmpeg... Trusty has no ffmpeg unless installing a backport package, however.

what's the respective ffmpeg/avconv commandline we can use to test this?

Here's a sample from live debug log on a test site:

wfShellExec: /bin/bash '/vagrant/mediawiki/includes/limit.sh' ''\''/usr/bin/ffmpeg'\'' -y -i '\''/srv/images/b/b4/MountTsukuba-Ropeway-aerial_lift-Japan-March5-2015.ogv'\'' -threads 1 -skip_threshold 0 -bufsize 6000k -rc_init_occupancy 4000 -qmin 1 -qmax 51 -vb '\''512000'\'' -vcodec libvpx -g '\''128'\'' -keyint_min '\''128'\'' -f webm -s 640x360 -an -pass '\''1'\'' -passlogfile '\''/tmp/transcode_360p.webm87085c68f73b-1.webm.log'\'' /dev/null 2>&1' 'MW_INCLUDE_STDERR=;MW_CPU_LIMIT=28800; MW_CGROUP='\'''\''; MW_MEM_LIMIT=2097152; MW_FILE_SIZE_LIMIT=3145728; MW_WALL_CLOCK_LIMIT=28800; MW_USE_LOG_PIPE=yes'
wfShellExec: /bin/bash '/vagrant/mediawiki/includes/limit.sh' ''\''/usr/bin/ffmpeg'\'' -y -i '\''/srv/images/b/b4/MountTsukuba-Ropeway-aerial_lift-Japan-March5-2015.ogv'\'' -threads 1 -skip_threshold 0 -bufsize 6000k -rc_init_occupancy 4000 -qmin 1 -qmax 51 -vb '\''512000'\'' -vcodec libvpx -g '\''128'\'' -keyint_min '\''128'\'' -f webm -s 640x360 -aq '\''1'\'' -ar '\''44100'\'' -acodec libvorbis  -pass '\''2'\'' -passlogfile '\''/tmp/transcode_360p.webm87085c68f73b-1.webm.log'\'' /tmp/transcode_360p.webm87085c68f73b-1.webm 2>&1' 'MW_INCLUDE_STDERR=;MW_CPU_LIMIT=28800; MW_CGROUP='\'''\''; MW_MEM_LIMIT=2097152; MW_FILE_SIZE_LIMIT=3145728; MW_WALL_CLOCK_LIMIT=28800; MW_USE_LOG_PIPE=yes'

Cleaned up that's something like:

wget 'https://upload.wikimedia.org/wikipedia/commons/b/b4/MountTsukuba-Ropeway-aerial_lift-Japan-March5-2015.ogv'

ffmpeg -y -i 'MountTsukuba-Ropeway-aerial_lift-Japan-March5-2015.ogv' -threads 1 -skip_threshold 0 -bufsize 6000k -rc_init_occupancy 4000 -qmin 1 -qmax 51 -vb 512000 -vcodec libvpx -g 128 -keyint_min 128 -f webm -s 640x360 -an -pass 1 -passlogfile /tmp/transcode_360p.webm87085c68f73b-1.webm.log /dev/null

ffmpeg -y -i MountTsukuba-Ropeway-aerial_lift-Japan-March5-2015.ogv -threads 1 -skip_threshold 0 -bufsize 6000k -rc_init_occupancy 4000 -qmin 1 -qmax 51 -vb 512000 -vcodec libvpx -g 128 -keyint_min 128 -f webm -s 640x360 -aq 1 -ar 44100 -acodec libvorbis  -pass 2 -passlogfile /tmp/transcode_360p.webm87085c68f73b-1.webm.log /tmp/transcode_360p.webm87085c68f73b-1.webm

thanks @brion! I gave this another try, though the input file is 1920x1080 and of course rescaling occurs. Anyways with the original input http://ogvjs-testing.wmflabs.org/images/transcoded/c/c8/Sneak_Preview_-_Wikipedia_VisualEditor.webm/Sneak_Preview_-_Wikipedia_VisualEditor.webm.360p.ogv and the same command line I can't see anything obviously lower quality with the latest ffmpeg.

since ffmpeg is uploaded to our internal repo for trusty, mediawiki-vagrant will pick the updated version by itself

vagrant@mediawiki-vagrant:~$ apt-cache policy ffmpeg
ffmpeg:
  Installed: (none)
  Candidate: 7:2.7.2-1~wmf1
  Version table:
     7:2.7.2-1~wmf1 0
        500 http://apt.wikimedia.org/wikimedia/ trusty-wikimedia/universe amd64 Packages

Excellent... TMH in Vagrant will presumably still be using avconv by default; lemme whip up a patch to the config to switch it to use ffmpeg. :)

Change 230078 had a related patch set uploaded (by Brion VIBBER):
Switch MW-Vagrant TMH to use ffmpeg instead of avconv

https://gerrit.wikimedia.org/r/230078

Change 230078 merged by jenkins-bot:
Switch MW-Vagrant TMH to use ffmpeg instead of avconv

https://gerrit.wikimedia.org/r/230078

brion claimed this task.

This is resolved with the switch to backported ffmpeg.