Note: this post has been published on 03/28 but has been originally written in September 2018 after Quibble 0.0.26 and never got published.

The last update about Quibble is from June 1st (Blog Post: Quibble in May), this is about updating on progress made over the summer.

Since the last update, Quibble version went from 0.0.17 to 0.0.26:

For --commands one pass them as shell snippets such as: --commands 'echo starting' 'phpunit' 'echo done'. A future version of Quibble would make it only accept a single argument though it can be repeated. Or in other terms, in the future one would have to use: --command 'echo starting' --command 'phpunit' --command 'echo done'.

The MediaWiki PHPUnit test suite to use is determined based on ZUUL_PROJECT. --phpunit-testsuite lets one explicitly set it, a use case is to run extensions tests for a change made to mediawiki/core and ensure it does not break extensions (ZUUL_PROJECT=mediawiki/core quibble --phpunit-testsuite=extensions mediawiki/extensions/BoilerPlate). On Wikimedia CI they are the wmf-quibble-* jobs.

You can get great speed up by using a tmpfs for the database. Create a tmpfs and then pass --db-dir to make use of it. With a Docker container one would do: docker run --tmpfs /workspace/db:size=320M quibble:latest --db-dir=/workspace/db.

In the future, I would like Quibble to be faster, it runs the commands in a serialized way and would be made faster by parallelizing at least some of the test commands (edit: done in 0.0.29).

Changelog for 0.0.17 to 0.0.26

• T196013 MediaWiki configuration injected by Quibble is now prepended at start of LocalSettings.php, that makes the configuration snippets available to wfLoadExtension() / wfLoadSkin().
• T197687 - Fix Chrome autoplay policy which prevented Qunit tests to run for Wikispeech https://goo.gl/xX8pDD
• T198171 - In Chrome do not rate limit history.pushState(), prevents some Qunit tests from passing since they overflow the limit.
• T195918
  • Enhance inline help for --run and --skip by grouping group them in a stages argument group.
  • New --skip=all to skip all tests
• T195084 T195918 - Support running any command inside the Quibble environment by using --commands (see below). They are run with a web server exposed (T203178).
• T22471 T196347 - rebuildLocalisationCache after update.php, fix locking issues when doing the first page request, multiple requests were racing over generating the localization cache.
• T200017 - Allow overriding the PHPUnit testsuite to run.
• Do not spawn a WebServer when running PHPUnit tests, its is only needed for Qunit and Selenium tests.
• Add a link to https://doc.wikimedia.org/quibble/ in the README.rst.
• T192132 - Quibble is now licensed under Apache 2.0
• T202710 - Xvfb no more listens on an unix socket.
• T200991 - Passing --dump-db-postrun will dump the content of the database to the log directory (--log-dir). Thanks @Pablo-WMDE
• Add support for Zuul cloner --branch and --project-branch, used to test MediaWiki-extensions-DonationInterface master branch against MediaWiki release branches.
• The environment variable TMPDIR set by Quibble is no more hardcoded to /tmp, it now follows the logic of Python tempfile.gettempdir().
• When running under Docker, default the log directory to be under the workspace instead of /log.
• Allow specifying database data directory with --db-dir (default is the temporary directory based on environment variable).