Quibble in summer
An update since June 1st 2018

Note: this post has been published on 03/28 but has been originally written in September 2018 after Quibble 0.0.26 and never got published.

The last update about Quibble is from June 1st (Blog Post: Quibble in May), this is about updating on progress made over the summer.

Since the last update, Quibble version went from 0.0.17 to 0.0.26:

For --commands one pass them as shell snippets such as: --commands 'echo starting' 'phpunit' 'echo done'. A future version of Quibble would make it only accept a single argument though it can be repeated. Or in other terms, in the future one would have to use: --command 'echo starting' --command 'phpunit' --command 'echo done'.

The MediaWiki PHPUnit test suite to use is determined based on ZUUL_PROJECT. --phpunit-testsuite lets one explicitly set it, a use case is to run extensions tests for a change made to mediawiki/core and ensure it does not break extensions (ZUUL_PROJECT=mediawiki/core quibble --phpunit-testsuite=extensions mediawiki/extensions/BoilerPlate). On Wikimedia CI they are the wmf-quibble-* jobs.

You can get great speed up by using a tmpfs for the database. Create a tmpfs and then pass --db-dir to make use of it. With a Docker container one would do: docker run --tmpfs /workspace/db:size=320M quibble:latest --db-dir=/workspace/db.

In the future, I would like Quibble to be faster, it runs the commands in a serialized way and would be made faster by parallelizing at least some of the test commands (edit: done in 0.0.29).

Changelog for 0.0.17 to 0.0.26

  • T196013 MediaWiki configuration injected by Quibble is now prepended at start of LocalSettings.php, that makes the configuration snippets available to wfLoadExtension() / wfLoadSkin().
  • T197687 - Fix Chrome autoplay policy which prevented Qunit tests to run for Wikispeech https://goo.gl/xX8pDD
  • T198171 - In Chrome do not rate limit history.pushState(), prevents some Qunit tests from passing since they overflow the limit.
  • T195918
    • Enhance inline help for --run and --skip by grouping group them in a stages argument group.
    • New --skip=all to skip all tests
  • T195084 T195918 - Support running any command inside the Quibble environment by using --commands (see below). They are run with a web server exposed (T203178).
  • T22471 T196347 - rebuildLocalisationCache after update.php, fix locking issues when doing the first page request, multiple requests were racing over generating the localization cache.
  • T200017 - Allow overriding the PHPUnit testsuite to run.
  • Do not spawn a WebServer when running PHPUnit tests, its is only needed for Qunit and Selenium tests.
  • Add a link to https://doc.wikimedia.org/quibble/ in the README.rst.
  • T192132 - Quibble is now licensed under Apache 2.0
  • T202710 - Xvfb no more listens on an unix socket.
  • T200991 - Passing --dump-db-postrun will dump the content of the database to the log directory (--log-dir). Thanks @Pablo-WMDE
  • Add support for Zuul cloner --branch and --project-branch, used to test MediaWiki-extensions-DonationInterface master branch against MediaWiki release branches.
  • The environment variable TMPDIR set by Quibble is no more hardcoded to /tmp, it now follows the logic of Python tempfile.gettempdir().
  • When running under Docker, default the log directory to be under the workspace instead of /log.
  • Allow specifying database data directory with --db-dir (default is the temporary directory based on environment variable).
Written by hashar on Mar 28 2019, 10:42 AM.
"Hungry Hippo" token, awarded by zeljkofilipin.

Event Timeline

As noted, I have actually wrote that in September 2018 just after Quibble 0.0.26. So it is a bit outdated but still worth publishing.

The next update is Blog Post: Quibble hibernated, it is time to flourish.