Create an integration test running Quibble with mediawiki/core
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	hashar
	Oct 9 2019, 5:55 PM

Description

I have cut 0.0.36 release way too fast, I had not even ran Quibble before cutting it. Eventually a couple faults happened which I caught too late forcing me to hotfix two follow up releases:

0.0.37 to fix lack of quibble/mediawiki/local_settings.php files in the installed Quibble
0.0.38 for $wgServer mismatching MW_SERVER - T235023

When sending a patch to Quibble repository, we should have a test / CI job that actually runs Quibble against mediawiki/core and run all tests. That will help catch up failures.

Details

Subject	Repo	Branch	Lines +/-
releasing: minor tweaks	integration/quibble	master	+6 -7
[WIP] Quibble self-test image	integration/config	master	+38 -0
Run Quibble fullrun job after tox	integration/config	master	+5 -4
Entry point for a CI fullrun	integration/quibble	master	+34 -0
Integration job for Quibble	integration/config	master	+29 -0
Define python deps in setup.cfg	integration/quibble	master	+10 -7
releasing: do run quibble before tagging	integration/quibble	master	+4 -0

Customize query in gerrit

Related Objects

Mentioned In: T238275: Unconference: [Workshop] CI counselling session
Mentioned Here: T235023: Selenium test "User should be able to log in @daily" fails with Quibble 0.0.37

Event Timeline

hashar created this task.Oct 9 2019, 5:55 PM

Change 541862 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/quibble@master] releasing: do run quibble before tagging

https://gerrit.wikimedia.org/r/541862

gerritbot added a project: Patch-For-Review.Oct 9 2019, 5:57 PM

hashar triaged this task as Medium priority.Oct 9 2019, 5:58 PM

hashar moved this task from Backlog to Enhancement on the Quibble board.

awight subscribed.Oct 10 2019, 11:19 AM

Jdforrester-WMF assigned this task to hashar.Oct 10 2019, 7:36 PM

Jdforrester-WMF moved this task from INBOX to Doing on the Release-Engineering-Team-TODO (201910) board.

Change 541862 merged by jenkins-bot:
[integration/quibble@master] releasing: do run quibble before tagging

https://gerrit.wikimedia.org/r/541862

Maintenance_bot removed a project: Patch-For-Review.Oct 11 2019, 10:10 AM

Jdforrester-WMF edited projects, added Release-Engineering-Team-TODO (201911); removed Release-Engineering-Team-TODO (201910).Nov 6 2019, 4:20 PM

Jdforrester-WMF moved this task from INBOX to Doing on the Release-Engineering-Team-TODO (201911) board.

I thought I could reuse the releng/quibble-stretch-php72 container but it lacks pip to do the installation, though it has easy_install through setuptools. Maybe that is sufficient to install Quibble from a local clone/fetch.

Jdforrester-WMF mentioned this in T238275: Unconference: [Workshop] CI counselling session.Nov 14 2019, 9:53 PM

Jdforrester-WMF edited projects, added Release-Engineering-Team-TODO (201912); removed Release-Engineering-Team-TODO (201911).Dec 2 2019, 7:31 PM

Jdforrester-WMF moved this task from INBOX to Doing on the Release-Engineering-Team-TODO (201912) board.

In T235118#5650604, @hashar wrote:

I thought I could reuse the releng/quibble-stretch-php72 container but it lacks pip to do the installation, though it has easy_install through setuptools. Maybe that is sufficient to install Quibble from a local clone/fetch.

In theory, it seems possible. For example, even as user nobody I'm able to do the following in that container:

HOME=/tmp python3 /quibble/setup.py install --user
HOME=/tmp easy_install-3.5 --user `cat /quibble/test-requirements.txt`

The setup.cfg wrapper around the main requirements makes it more challenging, I'm not sure if setup.py already includes a mechanism to extract this list and install. Anyway, doing it manually seems fine for now:

HOME=/tmp easy_install-3.5 --user extras six PyYAML GitPython
python3 -m quibble.cmd

Of course, this is all much simpler from a Dockerfile based on the PHP CI image, since we'll have root privileges.

thcipriani edited projects, added Release-Engineering-Team-TODO (2020-01 to 2020-03 (Q3)); removed Release-Engineering-Team-TODO (201912).Jan 7 2020, 10:52 PM

Jdforrester-WMF moved this task from INBOX to Maintenance on the Release-Engineering-Team-TODO (2020-01 to 2020-03 (Q3)) board.Jan 15 2020, 6:44 PM

hashar removed hashar as the assignee of this task.Mar 5 2020, 2:09 PM

Jdforrester-WMF edited projects, added Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)); removed Release-Engineering-Team-TODO (2020-01 to 2020-03 (Q3)).Apr 1 2020, 3:04 PM

Jdforrester-WMF moved this task from INBOX to Maintenance on the Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)) board.

awight claimed this task.Apr 16 2020, 7:33 PM

awight moved this task from Enhancement to In progress on the Quibble board.

Change 589438 had a related patch set uploaded (by Awight; owner: Awight):
[integration/config@master] [WIP] Quibble self-test image

https://gerrit.wikimedia.org/r/589438

gerritbot added a project: Patch-For-Review.Apr 16 2020, 10:04 PM

I could use advice about at least one detail: how should we clone quibble so that it doesn't overlap with /workspace/src ? A tiny image to move /workspace/src to /workspace/quibble ?

Change 604702 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/quibble@master] Define python deps in setup.cfg

https://gerrit.wikimedia.org/r/604702

In T235118#5708092, @awight wrote:
In T235118#5650604, @hashar wrote:

I thought I could reuse the releng/quibble-stretch-php72 container but it lacks pip to do the installation, though it has easy_install through setuptools. Maybe that is sufficient to install Quibble from a local clone/fetch.

In theory, it seems possible. For example, even as user nobody I'm able to do the following in that container:
HOME=/tmp python3 /quibble/setup.py install --user
HOME=/tmp easy_install-3.5 --user `cat /quibble/test-requirements.txt`
The setup.cfg wrapper around the main requirements makes it more challenging, I'm not sure if setup.py already includes a mechanism to extract this list and install. Anyway, doing it manually seems fine for now:
HOME=/tmp easy_install-3.5 --user extras six PyYAML GitPython
python3 -m quibble.cmd
Of course, this is all much simpler from a Dockerfile based on the PHP CI image, since we'll have root privileges.

With https://gerrit.wikimedia.org/r/#/c/integration/quibble/+/604702 we no more need the requirements.txt and they will be extra components. The idea is to not having to require to list the dependencies from the command lines or use tricks such as cat test-requirements.txt.

However, I could not find how to in.stall extra packages when using python3 setup.py install. Then, I don't think we need any test requirements, we just need to install Quibble and run it?

So:

HOME=/tmp python3 /quibble/setup.py install --user

And run it with something like:

/tmp/...whateverpath../bin/quibble

I could use advice about at least one detail: how should we clone quibble so that it doesn't overlap with /workspace/src ? A tiny image to move /workspace/src to /workspace/quibble ?

The jjb macro docker-ci-src-setup-simple would clone it to $WORKSPACE/src on the host, but when we then launch the releng/quibble container we can just mount it to wherever we want eg: docker run -v "$(pwd):/src/quibble" and install from there.

Change 604785 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Integration job for Quibble

https://gerrit.wikimedia.org/r/604785

Change 605939 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/quibble@master] Entry point for a CI fullrun

https://gerrit.wikimedia.org/r/605939

Change 604702 merged by jenkins-bot:
[integration/quibble@master] Define python deps in setup.cfg

https://gerrit.wikimedia.org/r/604702

Change 604785 merged by jenkins-bot:
[integration/config@master] Integration job for Quibble

https://gerrit.wikimedia.org/r/604785

Change 606492 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Run Quibble fullrun job after tox

https://gerrit.wikimedia.org/r/606492

Change 605939 merged by jenkins-bot:
[integration/quibble@master] Entry point for a CI fullrun

https://gerrit.wikimedia.org/r/605939

Change 606492 merged by jenkins-bot:
[integration/config@master] Run Quibble fullrun job after tox

https://gerrit.wikimedia.org/r/606492

hashar closed this task as Resolved.Jun 24 2020, 4:16 PM

🎉

Change 589438 abandoned by Hashar:
[integration/config@master] [WIP] Quibble self-test image

Reason:

https://gerrit.wikimedia.org/r/589438

Change 628299 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/quibble@master] releasing: minor tweaks

https://gerrit.wikimedia.org/r/628299

Change 628299 merged by jenkins-bot:
[integration/quibble@master] releasing: minor tweaks