If you pass --env ZUUL_PROJECT=mediawiki/extensions/MobileFrontend to docker and then --skip-zuul to quibble, it should kind of do what you want. One problem though is that quibble wants to use is own database, so it will clobber your LocalSettings.php.

@Legoktm, thank you! This kind of works ok but there's two issues I've noticed:

• A Git clean is executed in the project directory which blows away any new files added. This means I will have to commit any changes I want to test which isn't convenient for development.
• The second iteration fails (see below).

Here's the workflow I was trying:

git clone --bare https://gerrit.wikimedia.org/r/mediawiki/core ref/mediawiki/core.git
git clone --bare https://gerrit.wikimedia.org/r/mediawiki/vendor ref/mediawiki/vendor.git
git clone --bare https://gerrit.wikimedia.org/r/mediawiki/skins/Vector ref/mediawiki/skins/Vector.git

mkdir cache log src
docker run --env ZUUL_PROJECT=mediawiki/extensions/MobileFrontend -it --rm -u "$(id -u "$USER"):$(id -g "$USER")" \
  -v /etc/group:/etc/group:ro \
  -v /etc/passwd:/etc/passwd:ro \
  -v "$(pwd)"/cache:/cache \
  -v "$(pwd)"/log:/workspace/log \
  -v "$(pwd)"/ref:/srv/git:ro \
  -v "$(pwd)"/src:/workspace/src \
  docker-registry.wikimedia.org/releng/quibble-stretch:latest

# ^^^ it works. Now scribble in my local changes and iterate as wanted.
docker run --env ZUUL_PROJECT=mediawiki/extensions/MobileFrontend -it --rm -u "$(id -u "$USER"):$(id -g "$USER")" \
  -v /etc/group:/etc/group:ro \
  -v /etc/passwd:/etc/passwd:ro \
  -v "$(pwd)"/cache:/cache \
  -v "$(pwd)"/log:/workspace/log \
  -v "$(pwd)"/ref:/srv/git:ro \
  -v "$(pwd)"/src:/workspace/src \
  docker-registry.wikimedia.org/releng/quibble-stretch:latest --skip-zuul

# First iteration works!
# Run again and...
...
...
...
INFO:mw.maintenance.install:php maintenance/install.php --scriptpath= --dbtype=mysql --dbname=wikidb --dbuser=wikiuser --dbpass=secret --dbserver=localhost:/tmp/quibble-mysql-l9ro9wsa/socket --with-extensions --pass=testpass TestWiki WikiAdmin
PHP 7.0.27-0+deb9u1 is installed.
Found ImageMagick: /usr/bin/convert. Image thumbnailing will be enabled if you enable uploads.
Found the Git version control software: /usr/bin/git.
Using server URL "".
Warning: Your default directory for uploads (/workspace/src/images/) is not checked for vulnerability to arbitrary script execution during the CLI install.
Using the intl PECL extension for Unicode normalization.
The environment has been checked. You can install MediaWiki.
A LocalSettings.php file has been detected. To upgrade this installation, please run update.php instead

Traceback (most recent call last):
  File "/usr/local/bin/quibble", line 11, in <module>
    load_entry_point('quibble==0.0.0', 'console_scripts', 'quibble')()
  File "/usr/local/lib/python3.5/dist-packages/quibble/cmd.py", line 459, in main
    cmd.execute()
  File "/usr/local/lib/python3.5/dist-packages/quibble/cmd.py", line 368, in execute
    self.mw_install()
  File "/usr/local/lib/python3.5/dist-packages/quibble/cmd.py", line 228, in mw_install
    mwdir=self.mw_install_path
  File "/usr/local/lib/python3.5/dist-packages/quibble/mediawiki/maintenance.py", line 48, in install
    'Install failed with exit code: %s' % p.returncode)
Exception: Install failed with exit code: 1
INFO:backend.MySQL:Terminating MySQL

In T195084#4249719, @Niedzielski wrote:

• A Git clean is executed in the project directory which blows away any new files added. This means I will have to commit any changes I want to test which isn't convenient for development.

Hmm. I guess we could disable the git clean step if you pass --skip-zuul?

• The second iteration fails (see below).

That's because you already have a LocalSettings.php file. Deleting it after the first run should make it work. We could have quibble move the existing LocalSettings.php file to a safe place, run the installer, do the tests, and then restore the LocalSettings.php at the end?

In general quibble has been pretty optimized for CI usage so far, and not that much for local usage. Patches are definitely welcome :)

Change 447449 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/quibble@master] Run any shell command inside Quibble test env

https://gerrit.wikimedia.org/r/447449

https://gerrit.wikimedia.org/r/447449 would eventually let Quibble do the installation, spawn MySQL and then let one run any command in that context. Currently: --user-commands="cd extensions/MobileFrontend && somecommand. Pending changes based on reviews by others.

Change 447449 merged by jenkins-bot:
[integration/quibble@master] Run any command inside Quibble test env

https://gerrit.wikimedia.org/r/447449

A LocalSettings.php file has been detected. To upgrade this installation, please run update.php instead

That one is annoying. Quibble does not delete LocalSettings.php since I was afraid of eventually deleting a file coming from the user. But maybe we can back it up and delete it? Might need another task for that one.

EDIT: filled as T218647: Quibble should backup and delete LocalSettings.php

The process to reproduce is to first run quibble, then for the next iterations to always pass --skip-zuul --skip-deps which prevents running git again (which does git reset --hard / git clean) and skip npm/composer install which are slow. Instead, I would like to decouple quibble in two subcommands: one to create the source tree / install dependencies, another command to run the mediawiki installation/spawnbackends/run tests.

As for the original request, --commands lets one pass commands to be executed instead of the built-in tests (named 'stages').

For PHPUnit there is:

--phpunit-testsuite pattern
                        PHPUnit: filter which testsuite to run

Which is passed to mediawiki/core wrapper eg php tests/phpunit/phpunit.php --testsuite=XXXXX.

When:

• ZUUL_PROJECT starts with mediawiki/extensions/, the testsuite is set to extensions
• ZUUL_PROJECT starts with mediawiki/skins/, the testsuite is set to skins

Else it is not passed (unless --phpunit-testsuite pattern is passed to Quibble.

Available suites:

$ php tests/phpunit/phpunit.php --list-suites
Using PHP 7.0.33-0+deb9u3
PHPUnit 6.5.14 by Sebastian Bergmann and contributors.

Available test suite(s):
 - includes
 - languages
 - parsertests
 - skins
 - maintenance_suite
 - structure
 - tests
 - uploadfromurl
 - extensions
 - documentation

Maybe our phpunit wrapper could generate suite for each installed extensions/skins?