Page MenuHomePhabricator

Add local repository support to Quibble
Open, Needs TriagePublic

Description

Quibble and Docker support downloading a complete environment and running all tests. This task requests support to download a complete environment but allow for a local repository, such as MobileFrontend, to be overridden and to run only tests for that environment. The proposed syntax is as follows:

docker run -it --rm \
  -v "$(pwd)"/cache:/cache \
  -v "$(pwd)"/log:/workspace/log \
  -v "$(pwd)"/ref:/srv/git:ro \
  -v "$(pwd)"/src:/workspace/src \
  -v "$(pwd)"/vagrant/mediawiki/extensions/MobileFrontend:/workspace/src/mediawiki/extensions/MobileFrontend \
  docker-registry.wikimedia.org/releng/quibble-stretch:latest \
  --test extensions/MobileFrontend

In the above example, two changes occur:

  1. Use the local copy of MobileFrontend specified. Don't download this extension.
  2. Only run tests for MobileFrontend.

The use case is for testing local patches developed in a local Vagrant instance.

Details

Related Gerrit Patches:
integration/quibble : masterRun any command inside Quibble test env

Event Timeline

Legoktm added a subscriber: Legoktm.Jun 1 2018, 7:44 AM

If you pass --env ZUUL_PROJECT=mediawiki/extensions/MobileFrontend to docker and then --skip-zuul to quibble, it should kind of do what you want. One problem though is that quibble wants to use is own database, so it will clobber your LocalSettings.php.

@Legoktm, thank you! This kind of works ok but there's two issues I've noticed:

  • A Git clean is executed in the project directory which blows away any new files added. This means I will have to commit any changes I want to test which isn't convenient for development.
  • The second iteration fails (see below).

Here's the workflow I was trying:

git clone --bare https://gerrit.wikimedia.org/r/mediawiki/core ref/mediawiki/core.git
git clone --bare https://gerrit.wikimedia.org/r/mediawiki/vendor ref/mediawiki/vendor.git
git clone --bare https://gerrit.wikimedia.org/r/mediawiki/skins/Vector ref/mediawiki/skins/Vector.git

mkdir cache log src
docker run --env ZUUL_PROJECT=mediawiki/extensions/MobileFrontend -it --rm -u "$(id -u "$USER"):$(id -g "$USER")" \
  -v /etc/group:/etc/group:ro \
  -v /etc/passwd:/etc/passwd:ro \
  -v "$(pwd)"/cache:/cache \
  -v "$(pwd)"/log:/workspace/log \
  -v "$(pwd)"/ref:/srv/git:ro \
  -v "$(pwd)"/src:/workspace/src \
  docker-registry.wikimedia.org/releng/quibble-stretch:latest

# ^^^ it works. Now scribble in my local changes and iterate as wanted.
docker run --env ZUUL_PROJECT=mediawiki/extensions/MobileFrontend -it --rm -u "$(id -u "$USER"):$(id -g "$USER")" \
  -v /etc/group:/etc/group:ro \
  -v /etc/passwd:/etc/passwd:ro \
  -v "$(pwd)"/cache:/cache \
  -v "$(pwd)"/log:/workspace/log \
  -v "$(pwd)"/ref:/srv/git:ro \
  -v "$(pwd)"/src:/workspace/src \
  docker-registry.wikimedia.org/releng/quibble-stretch:latest --skip-zuul

# First iteration works!
# Run again and...
...
...
...
INFO:mw.maintenance.install:php maintenance/install.php --scriptpath= --dbtype=mysql --dbname=wikidb --dbuser=wikiuser --dbpass=secret --dbserver=localhost:/tmp/quibble-mysql-l9ro9wsa/socket --with-extensions --pass=testpass TestWiki WikiAdmin
PHP 7.0.27-0+deb9u1 is installed.
Found ImageMagick: /usr/bin/convert. Image thumbnailing will be enabled if you enable uploads.
Found the Git version control software: /usr/bin/git.
Using server URL "".
Warning: Your default directory for uploads (/workspace/src/images/) is not checked for vulnerability to arbitrary script execution during the CLI install.
Using the intl PECL extension for Unicode normalization.
The environment has been checked. You can install MediaWiki.
A LocalSettings.php file has been detected. To upgrade this installation, please run update.php instead

Traceback (most recent call last):
  File "/usr/local/bin/quibble", line 11, in <module>
    load_entry_point('quibble==0.0.0', 'console_scripts', 'quibble')()
  File "/usr/local/lib/python3.5/dist-packages/quibble/cmd.py", line 459, in main
    cmd.execute()
  File "/usr/local/lib/python3.5/dist-packages/quibble/cmd.py", line 368, in execute
    self.mw_install()
  File "/usr/local/lib/python3.5/dist-packages/quibble/cmd.py", line 228, in mw_install
    mwdir=self.mw_install_path
  File "/usr/local/lib/python3.5/dist-packages/quibble/mediawiki/maintenance.py", line 48, in install
    'Install failed with exit code: %s' % p.returncode)
Exception: Install failed with exit code: 1
INFO:backend.MySQL:Terminating MySQL
  • A Git clean is executed in the project directory which blows away any new files added. This means I will have to commit any changes I want to test which isn't convenient for development.

Hmm. I guess we could disable the git clean step if you pass --skip-zuul?

  • The second iteration fails (see below).

That's because you already have a LocalSettings.php file. Deleting it after the first run should make it work. We could have quibble move the existing LocalSettings.php file to a safe place, run the installer, do the tests, and then restore the LocalSettings.php at the end?

In general quibble has been pretty optimized for CI usage so far, and not that much for local usage. Patches are definitely welcome :)

Niedzielski closed this task as Invalid.Jun 11 2018, 1:31 PM

@Legoktm, yeah maybe I'm pushing to add complexity that shouldn't exist in Quibble. I'm going to close this task.

Legoktm reopened this task as Open.Jun 11 2018, 5:43 PM

@Legoktm, yeah maybe I'm pushing to add complexity that shouldn't exist in Quibble. I'm going to close this task.

Sorry, I was unclear. We *do* want this functionality, so that people can easily run and reproduce CI failures locally. I just meant that we really haven't gotten around to that yet.

Vvjjkkii renamed this task from Add local repository support to Quibble to bocaaaaaaa.Jul 1 2018, 1:08 AM
Vvjjkkii triaged this task as High priority.
Vvjjkkii updated the task description. (Show Details)
CommunityTechBot renamed this task from bocaaaaaaa to Add local repository support to Quibble.Jul 1 2018, 6:11 PM
CommunityTechBot raised the priority of this task from High to Needs Triage.
CommunityTechBot updated the task description. (Show Details)

Change 447449 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/quibble@master] Run any shell command inside Quibble test env

https://gerrit.wikimedia.org/r/447449

https://gerrit.wikimedia.org/r/447449 would eventually let Quibble do the installation, spawn MySQL and then let one run any command in that context. Currently: --user-commands="cd extensions/MobileFrontend && somecommand. Pending changes based on reviews by others.

Change 447449 merged by jenkins-bot:
[integration/quibble@master] Run any command inside Quibble test env

https://gerrit.wikimedia.org/r/447449

hashar added a comment.EditedMar 19 2019, 10:38 AM

A LocalSettings.php file has been detected. To upgrade this installation, please run update.php instead

That one is annoying. Quibble does not delete LocalSettings.php since I was afraid of eventually deleting a file coming from the user. But maybe we can back it up and delete it? Might need another task for that one.

EDIT: filled as T218647: Quibble should backup and delete LocalSettings.php


The process to reproduce is to first run quibble, then for the next iterations to always pass --skip-zuul --skip-deps which prevents running git again (which does git reset --hard / git clean) and skip npm/composer install which are slow. Instead, I would like to decouple quibble in two subcommands: one to create the source tree / install dependencies, another command to run the mediawiki installation/spawnbackends/run tests.


As for the original request, --commands lets one pass commands to be executed instead of the built-in tests (named 'stages').

For PHPUnit there is:

--phpunit-testsuite pattern
                        PHPUnit: filter which testsuite to run

Which is passed to mediawiki/core wrapper eg php tests/phpunit/phpunit.php --testsuite=XXXXX.

When:

  • ZUUL_PROJECT starts with mediawiki/extensions/, the testsuite is set to extensions
  • ZUUL_PROJECT starts with mediawiki/skins/, the testsuite is set to skins

Else it is not passed (unless --phpunit-testsuite pattern is passed to Quibble.

Available suites:

$ php tests/phpunit/phpunit.php --list-suites
Using PHP 7.0.33-0+deb9u3
PHPUnit 6.5.14 by Sebastian Bergmann and contributors.

Available test suite(s):
 - includes
 - languages
 - parsertests
 - skins
 - maintenance_suite
 - structure
 - tests
 - uploadfromurl
 - extensions
 - documentation

Maybe our phpunit wrapper could generate suite for each installed extensions/skins?