Writing blog is neither my job nor something that I enjoy, I am thus late in the Quibble updates. The last one Blog Post: Quibble in summer has been written in September 2018 and I forgot to publish it until now. You might want to read it first to get a glance about some nice changes that got implemented last summer.
I guess personal changes that happened in October and the traditional norther hemisphere winter hibernation kind of explain the delay (see note [ 1 ]). Now that spring is finally there ({{NPOV}}), it is time for another update.
Quibble went from 0.0.26 to 0.0.30 which I have cut just before starting this post. I wanted to highlight a few changes from an overall small change log:
- Use stronger password in Quibble related browser tests - T204569
- Parallelize ext/skin linter
- Parallelize mediawiki/core linter
- PHPUnit generates Junit results - T207841
- readme: how to reproduce a CI build - T200991
- doc: quibble-stretch no more has php
- mediawiki.d: Avoid vars that look like core or wmf names
- Drop /p from Gerrit clone URL - T218844
- Support to clone repositories in parallel - T211701
- Properly abort when git submodule processing fails - T198980
- mediawiki.d: Improve docs about dev settings and combine env sections
- mediawiki.d: Merge into one file
Parallelism [ 2 ]
The first inception of Quibble did not have much thoughts put into it with regard to speed. The main goal at the time was simply to gather all the complicated logic from CI shell scripts, Jenkins jobs shell snippets, python or javascript scripts all in one single command. That in turn made it easier to reproduce a build but with a serious limitation: commands are just run serially which is far from being optimum.
Quibble would now run the lint commands in parallel for both extensions/skins and mediawiki/core. Internally, it forks run composer test and npm test in parallel, that slightly speed up the time to get linting commands to complete.
Another annoyance is when testing multiple repositories together, preparing the git repositories could takes several minutes. An example is for an extension depending on several other extensions or the gated wmf-quibble-* jobs which run tests for several Wikimedia deployed extensions. Even when using a local cache of git repositories (--git-cache) the serially run git commands take a while. Quibble 0.0.30 learned --git-parallel to run the git commands in parallel. An example speed up using git cache, several repositories and a DSL connection:
| git-parallel | Duration |
|---|---|
| 16 | 30 seconds |
| 1 | 50 seconds |
The option defaults to 1 which retain the exact same behavior / code path as before. I invite you to try --git-parallel=8 for example and draw your own conclusion. Wikimedia CI will be updated once Quibble 0.0.30 is deployed.
Parallelism added by myself, @hashar, and got partly tracked in T211701.
Documentation
Some part of the documentation referred to a Wikimedia CI containers that were no more suitable for running tests due to refactoring. The documentation as thus been updated to use the proper containers: docker-registry.wikimedia.org/releng/quibble-stretch-php72 or docker-registry.wikimedia.org/releng/quibble-stretch-hhvm. -- @hashar
In August, Wikidata developers used Quibble to reproduce a test failure and they did the extra step to capture their session and document how to reproduce it. Thank you @Pablo-WMDE for leading this and @Tarrow, @Addshore, @Michael, @Ladsgroup for the reviews - T200991.
You can read the documentation online at:
Note: as of this writing, the CI git servers are NOT publicly reachable (git://contint1001.wikimedia.org and git://contint2001.wikimedia.org).
Submodule failures
Some extensions or skins might have submodules, however we never caught errors when they failed to process and kept continuing. That later causes tests to fail in non obvious way and caused several people to loose time recently. T198980
The reason is Quibble simply borrowed a legacy shell script to handle submodules and that script has been broken since its first introduction in 2014. It relied on the find command which still exit 0 even with -exec /bin/false. The reason is that although /bin/false exit code is 1, that simply causes find to consider the -exec predicate to be false, find thus abort processing further predicates but that is not an error.
The logic has been ported to pure python and now properly abort when git submodule fails. That also drop the requirement to have the find command available which might help on Windows. -- @hashar
Miscellaneous tweaks
The configuration injected by Quibble in LocalSettings.php is now a single file when it previously was made of several small PHP files glued together by shelling out to php. The inline comments have been improved. -- @Krinkle
MediaWiki installer uses a slightly stronger password (testwikijenkinspass) to accommodate for a security hardening in MediaWiki core itself. -- @Reedy T204569
The Gerrit URL to clone the canonical git repository from has been updated to catch up with a change in Gerrit. Updated r/p to simply /r. -- @Legoktm T218844
PHPUnit generates JUnit test results in the log directory, intended to be captured and interpreted by CI. -- @hashar T207841
footnotes
[ 1 ] Seasons are location based and a cultural agreement, they are quite interesting in their own. They are reversed in the Norther and Southern hemisphere, do not exist at the equator while in India they define six seasons. Thus when I refer to a winter hibernation, it really just reflect my own biased point of view.
[ 2 ] Parallelism is fun, I can never manage to write that word without mixing up the number of r or l for some reason. As a sideway note, my favorite sport to watch is parallel bars (enwiki).