Note: this post has been published on 03/28 but has been originally written in September 2018 after Quibble 0.0.26 and never got published.
The last update about Quibble is from June 1st (Blog Post: Quibble in May), this is about updating on progress made over the summer.
Since the last update, Quibble version went from 0.0.17 to 0.0.26:
For --commands one pass them as shell snippets such as: --commands 'echo starting' 'phpunit' 'echo done'. A future version of Quibble would make it only accept a single argument though it can be repeated. Or in other terms, in the future one would have to use: --command 'echo starting' --command 'phpunit' --command 'echo done'.
The MediaWiki PHPUnit test suite to use is determined based on ZUUL_PROJECT. --phpunit-testsuite lets one explicitly set it, a use case is to run extensions tests for a change made to mediawiki/core and ensure it does not break extensions (ZUUL_PROJECT=mediawiki/core quibble --phpunit-testsuite=extensions mediawiki/extensions/BoilerPlate). On Wikimedia CI they are the wmf-quibble-* jobs.
You can get great speed up by using a tmpfs for the database. Create a tmpfs and then pass --db-dir to make use of it. With a Docker container one would do: docker run --tmpfs /workspace/db:size=320M quibble:latest --db-dir=/workspace/db.
In the future, I would like Quibble to be faster, it runs the commands in a serialized way and would be made faster by parallelizing at least some of the test commands (edit: done in 0.0.29).
Changelog for 0.0.17 to 0.0.26
- T196013 MediaWiki configuration injected by Quibble is now prepended at start of LocalSettings.php, that makes the configuration snippets available to wfLoadExtension() / wfLoadSkin().
- T197687 - Fix Chrome autoplay policy which prevented Qunit tests to run for Wikispeech https://goo.gl/xX8pDD
- T198171 - In Chrome do not rate limit history.pushState(), prevents some Qunit tests from passing since they overflow the limit.
- Enhance inline help for --run and --skip by grouping group them in a stages argument group.
- New --skip=all to skip all tests
- T195084 T195918 - Support running any command inside the Quibble environment by using --commands (see below). They are run with a web server exposed (T203178).
- T22471 T196347 - rebuildLocalisationCache after update.php, fix locking issues when doing the first page request, multiple requests were racing over generating the localization cache.
- T200017 - Allow overriding the PHPUnit testsuite to run.
- Do not spawn a WebServer when running PHPUnit tests, its is only needed for Qunit and Selenium tests.
- Add a link to https://doc.wikimedia.org/quibble/ in the README.rst.
- T192132 - Quibble is now licensed under Apache 2.0
- T202710 - Xvfb no more listens on an unix socket.
- T200991 - Passing --dump-db-postrun will dump the content of the database to the log directory (--log-dir). Thanks @Pablo-WMDE
- Add support for Zuul cloner --branch and --project-branch, used to test MediaWiki-extensions-DonationInterface master branch against MediaWiki release branches.
- The environment variable TMPDIR set by Quibble is no more hardcoded to /tmp, it now follows the logic of Python tempfile.gettempdir().
- When running under Docker, default the log directory to be under the workspace instead of /log.
- Allow specifying database data directory with --db-dir (default is the temporary directory based on environment variable).
As noted, I have actually wrote that in September 2018 just after Quibble 0.0.26. So it is a bit outdated but still worth publishing.
The next update is Blog Post: Quibble hibernated, it is time to flourish.