Page MenuHomePhabricator
Create Task
Maniphest T197469

Migrate gated extensions jobs to Quibble/Docker
Closed, ResolvedPublic
Actions

Assigned To
hashar
Authored By
hashar
Jun 15 2018, 2:02 PM
Tags
Referenced Files
F22261571: quibble-gated-failures.txt
Jun 15 2018, 2:38 PM
F22261513: quibble-gated-2.jpg
Jun 15 2018, 2:09 PM
F22261511: quibble-gated-1.jpg
Jun 15 2018, 2:09 PM
Subscribers
Aklapper
cscott
hashar
Jdforrester-WMF
Stashbot

Description

We have extensions being tested together and running Nodepool jobs:

mediawiki-extensions-hhvm-jessie
mediawiki-extensions-php55-jessie
mediawiki-extensions-php70-jessie
mediawiki-extensions-qunit-jessie

They should be migrated to a new set of jobs. Implementation details to follow. A first pass is b493d4e558368ab6fe9995051c6d9e2992df8bf2
Extensions/skins switched with https://gerrit.wikimedia.org/r/c/integration/config/+/441068
Gated extensions on release branches got properly switched as well (T197933)

Test changes

phpunit failurehttps://gerrit.wikimedia.org/r/c/mediawiki/core/+/92621

Notes by @hashar 6/14

quibble-gated-1.jpg (2×3 px, 1 MB)
quibble-gated-2.jpg (3×2 px, 728 KB)

More complication with REL branches, a gated extension might not be in the tarball, in such a case we should not use vendor but composer when targeting REL branches. T189560#4413972

Details

Show related patches Customize query in gerrit

Revisions and Commits

rCICF integration-config
rCICFb493d4e55836 Quibble jobs for Wikimedia extensions

Related Objects
Search...

Event Timeline

hashar triaged this task as Medium priority.Jun 15 2018, 2:02 PM
hashar created this task.
hashar added a commit: rCICFb493d4e55836: Quibble jobs for Wikimedia extensions.
hashar updated the task description. (Show Details)Jun 15 2018, 2:09 PM
hashar mentioned this in T183512: [EPIC] Migrate Mediawiki jobs from Nodepool to Docker.
gerritbot subscribed.Jun 15 2018, 2:17 PM

Change 440537 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Fix JJB variables passed to wmf-quibble jobs

https://gerrit.wikimedia.org/r/440537

gerritbot added a project: Patch-For-Review.Jun 15 2018, 2:17 PM
gerritbot added a comment.Jun 15 2018, 2:21 PM

Change 440537 merged by jenkins-bot:
[integration/config@master] Fix JJB variables passed to wmf-quibble jobs

https://gerrit.wikimedia.org/r/440537

gerritbot added a comment.Jun 15 2018, 2:26 PM

Change 440539 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/puppet@production] ci: add some gated extensions to git cache

https://gerrit.wikimedia.org/r/440539

hashar added a comment.Jun 15 2018, 2:38 PM

As noted on b493d4e558368ab6fe9995051c6d9e2992df8bf2 :

Quibble runs a different testsuite depending on ZUUL_PROJECT:
core or vendor:

phpunit.php --exclude-group Database
phpunit.php --group Database extension or skin:
phpunit.php --testsuite [extensions|skins]

mediawiki-extensions jobs shell out to mw-run-phpunit-allexts.sh which
always run the extensions suite:

phpunit.php --testsuite extensions

Hence mediawiki/core patches will behave differently.

Good news: wmf-quibble-vendor-mysql-php70-docker passed when triggered from AbuseFilter extension. It ran --testsuite extensions or 15791 tests.

When the build is triggered by mediawiki/core, it runs phpunit.php --exclude-group Database which failed. 23225 tests total:

https://integration.wikimedia.org/ci/job/wmf-quibble-vendor-mysql-php70-docker/3/

quibble-gated-failures.txt35 KBDownload

Maybe because MediaWiki core tests are being run as well.

hashar moved this task from Backlog to In-progress on the Release-Engineering-Team (Kanban) board.Jun 19 2018, 10:06 AM
gerritbot added a comment.Jun 19 2018, 5:07 PM

Change 441068 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Filter core/vendor for gated Quibble job

https://gerrit.wikimedia.org/r/441068

hashar updated the task description. (Show Details)Jun 19 2018, 7:00 PM
gerritbot added a comment.Jun 19 2018, 8:27 PM

Change 441127 had a related patch set uploaded (by Hashar; owner: Thiemo Kreuz (WMDE)):
[mediawiki/extensions/Translate@wmf/1.32.0-wmf.8] Add "DEFAULT CHARSET=utf8" to translate_tmf table

https://gerrit.wikimedia.org/r/441127

hashar added a subtask: T197721: [Translate] Error: 1283 Column 'tmf_text' cannot be part of FULLTEXT index.Jun 19 2018, 8:35 PM

wmf-quibble jobs are blocked by Translate for the wmf/1.32.0-wmf.8 branch which is T197721. It passes on master / wmf.999 though !

hashar closed subtask T197721: [Translate] Error: 1283 Column 'tmf_text' cannot be part of FULLTEXT index as Resolved.Jun 20 2018, 7:08 AM
gerritbot added a comment.Jun 20 2018, 7:26 AM

Change 441127 merged by jenkins-bot:
[mediawiki/extensions/Translate@wmf/1.32.0-wmf.8] Add "DEFAULT CHARSET=utf8" to translate_tmf table

https://gerrit.wikimedia.org/r/441127

hashar updated the task description. (Show Details)Jun 20 2018, 7:39 AM
gerritbot added a comment.Jun 20 2018, 7:40 AM

Change 441068 merged by jenkins-bot:
[integration/config@master] Migrate gated job to Quibble

https://gerrit.wikimedia.org/r/441068

hashar updated the task description. (Show Details)Jun 20 2018, 11:19 AM
ReleaseTaggerBot added a project: MW-1.32-notes (WMF-deploy-2018-06-12 (1.32.0-wmf.8)).Jun 30 2018, 2:01 PM
• Vvjjkkii renamed this task from Migrate gated extensions jobs to Quibble/Docker to 2taaaaaaaa.Jul 1 2018, 1:02 AM
• Vvjjkkii removed hashar as the assignee of this task.
• Vvjjkkii raised the priority of this task from Medium to High.
• Vvjjkkii reopened subtask T197721: [Translate] Error: 1283 Column 'tmf_text' cannot be part of FULLTEXT index as Open.
• Vvjjkkii added projects: CheckUser, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), Tamil-Sites, Gamepress, Hashtags, Jade, KartoEditor, Language-2018-Apr-June, New-Editor-Experiences, Mail, TCB-Team (now WMDE-TechWish).
• Vvjjkkii updated the task description. (Show Details)
• Vvjjkkii removed subscribers: gerritbot, Aklapper.
JJMC89 renamed this task from 2taaaaaaaa to Migrate gated extensions jobs to Quibble/Docker.Jul 1 2018, 1:42 AM
JJMC89 assigned this task to hashar.
JJMC89 lowered the priority of this task from High to Medium.
JJMC89 removed projects: TCB-Team (now WMDE-TechWish), Mail, New-Editor-Experiences, Language-2018-Apr-June, KartoEditor, Jade, Hashtags, Gamepress, Tamil-Sites, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), CheckUser.
JJMC89 updated the task description. (Show Details)
JJMC89 added a subscriber: Aklapper.
CommunityTechBot closed subtask T197721: [Translate] Error: 1283 Column 'tmf_text' cannot be part of FULLTEXT index as Resolved.Jul 2 2018, 1:20 PM
gerritbot added a comment.Jul 3 2018, 10:45 AM

Change 443578 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] (WIP) Quibble job for release tarballs

https://gerrit.wikimedia.org/r/443578

gerritbot added a comment.Jul 3 2018, 12:41 PM

Change 443604 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Gated extensions now use Quibble on REL branches

https://gerrit.wikimedia.org/r/443604

hashar mentioned this in T197933: CI no more inject extensions dependencies on REL branches.Jul 3 2018, 12:43 PM
gerritbot added a comment.Jul 3 2018, 1:00 PM

Change 443604 merged by jenkins-bot:
[integration/config@master] Gated extensions now use Quibble on REL branches

https://gerrit.wikimedia.org/r/443604

gerritbot added a comment.Jul 3 2018, 1:11 PM

Change 443614 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Gated extensions were no more running tests on REL

https://gerrit.wikimedia.org/r/443614

gerritbot added a comment.Jul 3 2018, 1:15 PM

Change 443614 merged by jenkins-bot:
[integration/config@master] Gated extensions were no more running tests on REL

https://gerrit.wikimedia.org/r/443614

hashar reopened subtask T197721: [Translate] Error: 1283 Column 'tmf_text' cannot be part of FULLTEXT index as Open.Jul 3 2018, 1:30 PM
hashar updated the task description. (Show Details)Jul 3 2018, 1:33 PM
hashar added a subtask: T197933: CI no more inject extensions dependencies on REL branches.Jul 3 2018, 1:39 PM
gerritbot added a comment.Jul 3 2018, 2:14 PM

Change 443636 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Remove legacy jobs from gated extensions

https://gerrit.wikimedia.org/r/443636

gerritbot added a comment.Jul 3 2018, 2:20 PM

Change 443636 merged by jenkins-bot:
[integration/config@master] Remove legacy jobs from gated extensions

https://gerrit.wikimedia.org/r/443636

hashar added a comment.Jul 3 2018, 2:24 PM

Extensions fully migrated.

Left to do is to migrate mediawiki/core and vendor

Jdforrester-WMF added a comment.Jul 3 2018, 4:10 PM
In T197469#4394294, @hashar wrote:

Extensions fully migrated.

Left to do is to migrate mediawiki/core and vendor

Amazing. Kudos!

gerritbot added a comment.Jul 5 2018, 8:48 AM

Change 443916 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Update Quibble QA report, gate is migrated

https://gerrit.wikimedia.org/r/443916

gerritbot added a comment.Jul 5 2018, 8:50 AM

Change 443916 merged by jenkins-bot:
[integration/config@master] Update Quibble QA report, gate is migrated

https://gerrit.wikimedia.org/r/443916

hashar mentioned this in T189560: mediawiki/vendor REL1_* no longer ship dependencies for wmf extensions that are not in the mediawiki tarball.Jul 10 2018, 9:13 PM
hashar updated the task description. (Show Details)
hashar mentioned this in T200017: Quibble should be able to be forced to run extensions testsuite when triggered from mediawiki/core.Jul 19 2018, 3:10 PM
gerritbot added a comment.Jul 27 2018, 11:48 AM

Change 448477 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] docker: quibble 0.0.21 + rebuild images

https://gerrit.wikimedia.org/r/448477

gerritbot added a comment.Jul 27 2018, 12:04 PM

Change 448477 merged by jenkins-bot:
[integration/config@master] docker: quibble 0.0.21 + rebuild images

https://gerrit.wikimedia.org/r/448477

gerritbot added a comment.Aug 1 2018, 10:56 AM

Change 449696 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Replace mediawiki-extensions with Quibble

https://gerrit.wikimedia.org/r/449696

gerritbot added a comment.Aug 1 2018, 2:56 PM

Change 449696 merged by jenkins-bot:
[integration/config@master] Replace mediawiki-extensions with Quibble

https://gerrit.wikimedia.org/r/449696

Stashbot subscribed.Aug 1 2018, 2:56 PM

Mentioned in SAL (#wikimedia-releng) [2018-08-01T14:56:56Z] <hashar> Phasing out mediawiki-extensions jobs in favor of wmf-quibble ones | T197469

hashar closed subtask T200017: Quibble should be able to be forced to run extensions testsuite when triggered from mediawiki/core as Resolved.Aug 1 2018, 3:08 PM
hashar closed this task as Resolved.Aug 1 2018, 3:19 PM

mediawiki-extensions jobs are gone. The last use case was running extensions tests from a patch triggered by mediawiki/core or vendor. That has been done by force passing --testsuite extensions to PHPUnit, a feature added in Quibble via T200017.

They are finally gone \o/

hashar closed subtask T197721: [Translate] Error: 1283 Column 'tmf_text' cannot be part of FULLTEXT index as Resolved.Aug 20 2018, 9:50 AM
cscott subscribed.EditedOct 4 2018, 5:15 PM

This bug is referenced from https://github.com/wikimedia/integration-config/blob/41e1e9a25d416e20ef4d236d5bff68c1e815538a/jjb/mediawiki.yaml#L265 :

# We do not run mediawiki/core tests with extensions installed
# https://phabricator.wikimedia.org/T197469#4293142

...but why?

Not running core tests with extensions installed means that extensions can break core tests, like happened in https://gerrit.wikimedia.org/r/c/mediawiki/core/+/463383 for instance.

EDIT: new bug filed: T206243: Core tests aren't run with extensions installed

cscott mentioned this in T206243: Core tests aren't run with extensions installed.Oct 4 2018, 5:21 PM
gerritbot added a comment.Jan 8 2019, 8:19 PM

Change 440539 merged by Dzahn:
[operations/puppet@production] ci: add some gated extensions to git cache

https://gerrit.wikimedia.org/r/440539

gerritbot added a comment.Apr 5 2019, 12:16 PM

Change 443578 abandoned by Hashar:
(WIP) Quibble job for release tarballs

https://gerrit.wikimedia.org/r/443578

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits