Migrate gated extensions jobs to Quibble/Docker
Closed, ResolvedPublic

Description

We have extensions being tested together and running Nodepool jobs:

mediawiki-extensions-hhvm-jessie
mediawiki-extensions-php55-jessie
mediawiki-extensions-php70-jessie
mediawiki-extensions-qunit-jessie

They should be migrated to a new set of jobs. Implementation details to follow. A first pass is b493d4e558368ab6fe9995051c6d9e2992df8bf2
Extensions/skins switched with https://gerrit.wikimedia.org/r/c/integration/config/+/441068
Gated extensions on release branches got properly switched as well (T197933)

Test changes

Notes by @hashar 6/14

More complication with REL branches, a gated extension might not be in the tarball, in such a case we should not use vendor but composer when targeting REL branches. T189560#4413972

Related Objects

hashar created this task.Jun 15 2018, 2:02 PM
hashar triaged this task as Normal priority.

Change 440537 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Fix JJB variables passed to wmf-quibble jobs

https://gerrit.wikimedia.org/r/440537

Change 440537 merged by jenkins-bot:
[integration/config@master] Fix JJB variables passed to wmf-quibble jobs

https://gerrit.wikimedia.org/r/440537

Change 440539 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/puppet@production] ci: add some gated extensions to git cache

https://gerrit.wikimedia.org/r/440539

As noted on b493d4e558368ab6fe9995051c6d9e2992df8bf2 :


Quibble runs a different testsuite depending on ZUUL_PROJECT:
core or vendor:

phpunit.php --exclude-group Database
phpunit.php --group Database extension or skin:
phpunit.php --testsuite [extensions|skins]

mediawiki-extensions jobs shell out to mw-run-phpunit-allexts.sh which
always run the extensions suite:

phpunit.php --testsuite extensions

Hence mediawiki/core patches will behave differently.


Good news: wmf-quibble-vendor-mysql-php70-docker passed when triggered from AbuseFilter extension. It ran --testsuite extensions or 15791 tests.

When the build is triggered by mediawiki/core, it runs phpunit.php --exclude-group Database which failed. 23225 tests total:

https://integration.wikimedia.org/ci/job/wmf-quibble-vendor-mysql-php70-docker/3/

Maybe because MediaWiki core tests are being run as well.

Change 441068 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Filter core/vendor for gated Quibble job

https://gerrit.wikimedia.org/r/441068

hashar updated the task description. (Show Details)Jun 19 2018, 7:00 PM

Change 441127 had a related patch set uploaded (by Hashar; owner: Thiemo Kreuz (WMDE)):
[mediawiki/extensions/Translate@wmf/1.32.0-wmf.8] Add "DEFAULT CHARSET=utf8" to translate_tmf table

https://gerrit.wikimedia.org/r/441127

wmf-quibble jobs are blocked by Translate for the wmf/1.32.0-wmf.8 branch which is T197721. It passes on master / wmf.999 though !

Change 441127 merged by jenkins-bot:
[mediawiki/extensions/Translate@wmf/1.32.0-wmf.8] Add "DEFAULT CHARSET=utf8" to translate_tmf table

https://gerrit.wikimedia.org/r/441127

hashar updated the task description. (Show Details)Jun 20 2018, 7:39 AM

Change 441068 merged by jenkins-bot:
[integration/config@master] Migrate gated job to Quibble

https://gerrit.wikimedia.org/r/441068

hashar updated the task description. (Show Details)Jun 20 2018, 11:19 AM
Vvjjkkii renamed this task from Migrate gated extensions jobs to Quibble/Docker to 2taaaaaaaa.
Vvjjkkii raised the priority of this task from Normal to High.
Vvjjkkii removed hashar as the assignee of this task.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed subscribers: gerritbot, Aklapper.
JJMC89 renamed this task from 2taaaaaaaa to Migrate gated extensions jobs to Quibble/Docker.Jul 1 2018, 1:42 AM
JJMC89 lowered the priority of this task from High to Normal.
JJMC89 assigned this task to hashar.
JJMC89 updated the task description. (Show Details)
JJMC89 added a subscriber: Aklapper.

Change 443578 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] (WIP) Quibble job for release tarballs

https://gerrit.wikimedia.org/r/443578

Change 443604 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Gated extensions now use Quibble on REL branches

https://gerrit.wikimedia.org/r/443604

Change 443604 merged by jenkins-bot:
[integration/config@master] Gated extensions now use Quibble on REL branches

https://gerrit.wikimedia.org/r/443604

Change 443614 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Gated extensions were no more running tests on REL

https://gerrit.wikimedia.org/r/443614

Change 443614 merged by jenkins-bot:
[integration/config@master] Gated extensions were no more running tests on REL

https://gerrit.wikimedia.org/r/443614

hashar updated the task description. (Show Details)Jul 3 2018, 1:33 PM

Change 443636 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Remove legacy jobs from gated extensions

https://gerrit.wikimedia.org/r/443636

Change 443636 merged by jenkins-bot:
[integration/config@master] Remove legacy jobs from gated extensions

https://gerrit.wikimedia.org/r/443636

hashar added a comment.Jul 3 2018, 2:24 PM

Extensions fully migrated.

Left to do is to migrate mediawiki/core and vendor

Extensions fully migrated.

Left to do is to migrate mediawiki/core and vendor

Amazing. Kudos!

Change 443916 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Update Quibble QA report, gate is migrated

https://gerrit.wikimedia.org/r/443916

Change 443916 merged by jenkins-bot:
[integration/config@master] Update Quibble QA report, gate is migrated

https://gerrit.wikimedia.org/r/443916

Change 448477 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] docker: quibble 0.0.21 + rebuild images

https://gerrit.wikimedia.org/r/448477

Change 448477 merged by jenkins-bot:
[integration/config@master] docker: quibble 0.0.21 + rebuild images

https://gerrit.wikimedia.org/r/448477

Change 449696 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Replace mediawiki-extensions with Quibble

https://gerrit.wikimedia.org/r/449696

Change 449696 merged by jenkins-bot:
[integration/config@master] Replace mediawiki-extensions with Quibble

https://gerrit.wikimedia.org/r/449696

Mentioned in SAL (#wikimedia-releng) [2018-08-01T14:56:56Z] <hashar> Phasing out mediawiki-extensions jobs in favor of wmf-quibble ones | T197469

hashar closed this task as Resolved.Aug 1 2018, 3:19 PM

mediawiki-extensions jobs are gone. The last use case was running extensions tests from a patch triggered by mediawiki/core or vendor. That has been done by force passing --testsuite extensions to PHPUnit, a feature added in Quibble via T200017.

They are finally gone \o/

cscott added a subscriber: cscott.EditedOct 4 2018, 5:15 PM

This bug is referenced from https://github.com/wikimedia/integration-config/blob/41e1e9a25d416e20ef4d236d5bff68c1e815538a/jjb/mediawiki.yaml#L265 :

# We do not run mediawiki/core tests with extensions installed
# https://phabricator.wikimedia.org/T197469#4293142

...but why?

Not running core tests with extensions installed means that extensions can break core tests, like happened in https://gerrit.wikimedia.org/r/c/mediawiki/core/+/463383 for instance.

EDIT: new bug filed: T206243: Core tests aren't run with extensions installed