Page MenuHomePhabricator
Create Task
Maniphest T119139

[keyresult] Migrate php (Zend and HHVM) CI jobs to Nodepool
Closed, ResolvedPublic
Actions

Details

SubjectRepoBranchLines +/-
Mediawiki core/vendor PHPUnit jobs to Nodepoolintegration/configmaster+87 -17
dib: misc packages for MediaWiki testingintegration/configmaster+6 -0
dib: add contint::packages::phpintegration/configmaster+2 -0
Drop composer jobs from permanent slavesintegration/configmaster+11 -11
Migrate remaining composer jobs to Nodepoolintegration/configmaster+8 -10
Switch experimental composer jobs to Nodepoolintegration/configmaster+15 -13
Pass composer-test55 jobs to Nodepool instancesintegration/configmaster+5 -7
Switch composer packages to Nodepoolintegration/configmaster+9 -13
Composer jobs on Nodepool instancesintegration/configmaster+83 -0
dib: composer and Zend PHP for mw on Trustyintegration/configmaster+7 -0
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedhasharT119138 [keyresult] Migrate majority of CI jobs to Nodepool (part 2)
 ResolvedhasharT119139 [keyresult] Migrate php (Zend and HHVM) CI jobs to Nodepool
 ResolvedJoeT125821 Provide a HHVM package for jessie-wikimedia matching version of trusty-wikimedia
 ResolvedJoeT86096 Switch HAT appservers to trusty's ICU (or newer)
 ResolvedJoeT78765 Convert jobrunners to HHVM
 ResolvedJoeT84842 Convert eqiad imagescalers to HHVM, Trusty
 ResolvedtstarlingT91468 HHVM with FastCGI does not support streaming output
 ResolvedJoeT93194 Create an HHVM 3.6.0 package, adding Tim's streaming patch
 ResolvedkaldariT58041 updateCollation.php script prohibitively slow for very large wikis
 ResolvedjcrespoT130692 Add new indexes from eec016ece6d2b30addcdf3d3efcc2ba59b10e858 to production databases
 ResolvedVolansT128353 Switchover to new s3 master
 ResolvedPRODUCTION ERRORaaronT126436 Spikes of mediawiki in read only for job runners after altering the s2 slaves topology
 InvalidNoneT126632 Scap should restart job runners to pick up new config
 DeclinedNoneT103786 Investigate installing php5.3 on trusty and/or debian instance
 ResolvedLegoktmT126211 Make /usr/bin/php a wrapper that picks the right PHP version on CI slaves
 ResolvedhasharT128092 Provide composer on the nodepool servers so OOjs UI can use it in the npm job
 ResolvedhasharT133203 Attempt to provide a Trusty image for Nodepool
Restricted Task
 ResolvedhasharT126594 Disable HHVM fcgi server on CI slaves
 ResolvedoriT126658 /usr/lib/x86_64-linux-gnu/hhvm/extensions/current/luasandbox.so no such file or directory
 ResolvedhasharT133976 Migrate mediawiki-core-phpcs job to Nodepool
 ResolvedhasharT133911 Bump quota of Nodepool instances (contintcloud tenant)
 ResolvedhasharT139771 Identify metric (or metrics) that gives a useful indication of user-perceived (Wikimedia developer) service of CI
 ResolvedhasharT134177 wikimedia/slimapp fails composer-package-php55-trusty
 InvalidNoneT134182 labs/tools/stewardbots and translatewiki repos do not pass 'composer validate'
 DeclinedNoneT134381 Migrate PHP extensions building jobs to Nodepool
 ResolvedhasharT135001 Migrate PHPUnit MediaWiki core jobs to Nodepool
 ResolvedhasharT137199 Migrate mwext-testextension* jobs to nodepool

Event Timeline

greg created this task.Nov 19 2015, 10:53 PM
greg raised the priority of this task from to Medium.
greg updated the task description. (Show Details)
greg added a project: Continuous-Integration-Config.
greg added subscribers: hashar, Aklapper, greg.
hashar added projects: Continuous-Integration-Scaling, NewFunctionality-Worktype.Nov 20 2015, 2:49 PM

Macro whereismypopcorn:

greg added a project: releng-201516-q3.Dec 2 2015, 9:35 PM
greg set Security to None.
greg renamed this task from Migrate php composer (Zend and HHVM) CI jobs to Nodepool to [keyresult] Migrate php composer (Zend and HHVM) CI jobs to Nodepool.Jan 20 2016, 12:14 AM
hashar added a subtask: T125821: Provide a HHVM package for jessie-wikimedia matching version of trusty-wikimedia.Feb 4 2016, 2:29 PM

We first need bits to be able to select which flavor of PHP (Zend 5.3, Zend 5.5 or HHVM). We are not there yet. Some attempt has been made to switch to Zend 5.5 (but still on Trusty permanent slave). That is a work in progress.

Then we can generalize to use HHVM. We will then need a package on Jessie (low prio blocker is T125821) before being able to migrate to Nodepool instances.

hashar added a subtask: T103786: Investigate installing php5.3 on trusty and/or debian instance.Feb 11 2016, 9:36 PM
hashar added a subscriber: Legoktm.
In T119139#1997906, @hashar wrote:

We first need bits to be able to select which flavor of PHP (Zend 5.3, Zend 5.5 or HHVM). We are not there yet. Some attempt has been made to switch to Zend 5.5 (but still on Trusty permanent slave). That is a work in progress.

Aced by @Legoktm with T126211: Make /usr/bin/php a wrapper that picks the right PHP version on CI slaves

Paladox added a subscriber: Paladox.Feb 24 2016, 1:11 AM
Paladox added a comment.Mar 18 2016, 10:52 AM

@hashar php5 for Jessie https://packages.debian.org/jessie/php5

Jdforrester-WMF added a subtask: T128092: Provide composer on the nodepool servers so OOjs UI can use it in the npm job.Mar 18 2016, 11:29 PM
Ricordisamoa added a subscriber: Ricordisamoa.Mar 19 2016, 12:54 AM
hashar added a comment.Apr 1 2016, 6:52 PM

Status update

Prerequisite tasks have been accomplished such as introducing the PHP_BIN based wrapper and migrating repositories to use composer as an entry point.

Jessie provides Zend 5.6. For MediaWiki we will need Zend 5.5 and HHVM both having tasks and being blocker of this.

greg added a project: releng-201516-q4.Apr 1 2016, 8:59 PM

Added our q4 project to this

Paladox added a comment.Apr 1 2016, 9:09 PM

Maybe we can update php using the nodepool image and also install composer with it too.

hashar changed the status of subtask T125821: Provide a HHVM package for jessie-wikimedia matching version of trusty-wikimedia from Open to Stalled.Apr 20 2016, 7:34 PM
hashar created subtask T133203: Attempt to provide a Trusty image for Nodepool.Apr 20 2016, 7:37 PM
hashar added subscribers: akosiaris, Joe.Apr 25 2016, 6:38 PM

@Joe provided HHVM on Jessie (T125821) and on that task @akosiaris pointed out that libicu would be different on Jessie. He suggest to instead use Trusty which would be closer to what we run in production.

I did not really want to maintain two distributions but eventually providing a Trusty image was "just" a one day sprint T133203.

I will start crafting a few experimental jobs and refine as needed. We should then be able to migrate both HHVM and Zend 5.5 jobs to Nodepool instances.

hashar closed subtask T133203: Attempt to provide a Trusty image for Nodepool as Resolved.Apr 25 2016, 7:45 PM
gerritbot added a subscriber: gerritbot.Apr 25 2016, 8:08 PM

Change 285236 had a related patch set uploaded (by Hashar):
dib: composer and Zend PHP for mw on Trusty

https://gerrit.wikimedia.org/r/285236

gerritbot added a project: Patch-For-Review.Apr 25 2016, 8:08 PM
hashar added a subtask: T126594: Disable HHVM fcgi server on CI slaves.Apr 25 2016, 8:22 PM
gerritbot added a comment.Apr 26 2016, 4:55 PM

Change 285236 merged by jenkins-bot:
dib: composer and Zend PHP for mw on Trusty

https://gerrit.wikimedia.org/r/285236

hashar added a comment.Apr 27 2016, 10:42 AM

Status update

The Trusty image is provided as of T133203

I have added Zend php5 and composer but listed actions on T128092#2240335 (and below). Seems composer works fine there.

hashar added a subtask: T126211: Make /usr/bin/php a wrapper that picks the right PHP version on CI slaves.Apr 27 2016, 10:42 AM

We need our PHP_BIN wrapper script to be merged so it can be included on Nodepool image. That is T126211 which is just pending merge of a puppet patch.

gerritbot added a comment.Apr 27 2016, 1:48 PM

Change 285642 had a related patch set uploaded (by Hashar):
Composer jobs on Nodepool instances

https://gerrit.wikimedia.org/r/285642

gerritbot added a comment.Apr 27 2016, 3:35 PM

Change 285642 merged by jenkins-bot:
Composer jobs on Nodepool instances

https://gerrit.wikimedia.org/r/285642

hashar claimed this task.Apr 27 2016, 9:29 PM
hashar moved this task from Backlog to In progress on the Continuous-Integration-Config board.
hashar moved this task from Backlog to In-progress on the Continuous-Integration-Scaling board.
hashar closed subtask T126211: Make /usr/bin/php a wrapper that picks the right PHP version on CI slaves as Resolved.Apr 28 2016, 12:34 PM
hashar added a comment.Apr 28 2016, 3:15 PM

Did progress on provisioning PHP:

DistroZendHHVM
TrustyYesYes
JessieNOYes
hashar closed subtask T128092: Provide composer on the nodepool servers so OOjs UI can use it in the npm job as Resolved.Apr 28 2016, 3:23 PM
hashar renamed this task from [keyresult] Migrate php composer (Zend and HHVM) CI jobs to Nodepool to [keyresult] Migrate php (Zend and HHVM) CI jobs to Nodepool.Apr 29 2016, 10:10 AM
hashar created subtask T133976: Migrate mediawiki-core-phpcs job to Nodepool.Apr 29 2016, 10:13 AM
hashar added a subtask: T133911: Bump quota of Nodepool instances (contintcloud tenant).Apr 29 2016, 10:24 AM
JanZerebecki merged a task: T132681: mwext-testextension-hhvm-composer build timed out (after 30 minutes).Apr 30 2016, 11:06 PM
JanZerebecki mentioned this in T132681: mwext-testextension-hhvm-composer build timed out (after 30 minutes).
JanZerebecki added subscribers: JanZerebecki, adrianheine.
gerritbot added a comment.May 2 2016, 2:01 PM

Change 286433 had a related patch set uploaded (by Hashar):
Switches composer packages to Nodepool

https://gerrit.wikimedia.org/r/286433

hashar created subtask T134177: wikimedia/slimapp fails composer-package-php55-trusty.May 2 2016, 2:39 PM
gerritbot added a comment.May 2 2016, 2:58 PM

Change 286433 merged by jenkins-bot:
Switch composer packages to Nodepool

https://gerrit.wikimedia.org/r/286433

hashar added a subtask: T134182: labs/tools/stewardbots and translatewiki repos do not pass 'composer validate'.May 2 2016, 3:14 PM
hashar closed subtask T134182: labs/tools/stewardbots and translatewiki repos do not pass 'composer validate' as Invalid.May 2 2016, 3:24 PM
gerritbot added a comment.May 2 2016, 3:28 PM

Change 286452 had a related patch set uploaded (by Hashar):
Pass composer-test55 jobs to Nodepool instances

https://gerrit.wikimedia.org/r/286452

gerritbot added a comment.May 2 2016, 3:29 PM

Change 286452 merged by jenkins-bot:
Pass composer-test55 jobs to Nodepool instances

https://gerrit.wikimedia.org/r/286452

gerritbot added a comment.May 2 2016, 3:41 PM

Change 286460 had a related patch set uploaded (by Hashar):
Switch experimental composer jobs to Nodepool

https://gerrit.wikimedia.org/r/286460

gerritbot added a comment.May 2 2016, 3:44 PM

Change 286462 had a related patch set uploaded (by Hashar):
Migrate remaining composer jobs to Nodepool

https://gerrit.wikimedia.org/r/286462

gerritbot added a comment.May 2 2016, 3:46 PM

Change 286460 merged by jenkins-bot:
Switch experimental composer jobs to Nodepool

https://gerrit.wikimedia.org/r/286460

gerritbot added a comment.May 2 2016, 7:08 PM

Change 286497 had a related patch set uploaded (by Hashar):
(WIP) Mediawiki PHPUnit to Nodepool (WIP)

https://gerrit.wikimedia.org/r/286497

gerritbot added a comment.May 3 2016, 7:03 AM

Change 286462 merged by jenkins-bot:
Migrate remaining composer jobs to Nodepool

https://gerrit.wikimedia.org/r/286462

hashar closed subtask T134177: wikimedia/slimapp fails composer-package-php55-trusty as Resolved.May 3 2016, 7:38 AM
gerritbot added a comment.May 3 2016, 8:53 AM

Change 286602 had a related patch set uploaded (by Hashar):
Drop composer jobs from permanent slaves

https://gerrit.wikimedia.org/r/286602

gerritbot added a comment.May 3 2016, 8:55 AM

Change 286602 merged by jenkins-bot:
Drop composer jobs from permanent slaves

https://gerrit.wikimedia.org/r/286602

hashar closed subtask T133976: Migrate mediawiki-core-phpcs job to Nodepool as Resolved.May 4 2016, 12:30 PM
hashar created subtask T134381: Migrate PHP extensions building jobs to Nodepool.May 4 2016, 1:23 PM
hashar closed subtask T126594: Disable HHVM fcgi server on CI slaves as Resolved.May 9 2016, 10:03 AM
gerritbot added a comment.May 11 2016, 1:17 PM

Change 288187 had a related patch set uploaded (by Hashar):
dib: add contint::packages::php

https://gerrit.wikimedia.org/r/288187

gerritbot added a comment.May 11 2016, 1:21 PM

Change 288187 merged by jenkins-bot:
dib: add contint::packages::php

https://gerrit.wikimedia.org/r/288187

Stashbot added a subscriber: Stashbot.May 11 2016, 1:33 PM

Mentioned in SAL [2016-05-11T13:33:52Z] <hashar> Added contint::packages::php to Nodepool images T119139

gerritbot added a comment.May 11 2016, 2:17 PM

Change 288203 had a related patch set uploaded (by Hashar):
dib: misc packages for MediaWiki testing

https://gerrit.wikimedia.org/r/288203

gerritbot added a comment.May 11 2016, 2:19 PM

Change 288203 merged by jenkins-bot:
dib: misc packages for MediaWiki testing

https://gerrit.wikimedia.org/r/288203

hashar created subtask T135001: Migrate PHPUnit MediaWiki core jobs to Nodepool.May 11 2016, 2:36 PM
gerritbot added a comment.May 12 2016, 3:02 PM

Change 286497 merged by jenkins-bot:
Mediawiki core/vendor PHPUnit jobs to Nodepool

https://gerrit.wikimedia.org/r/286497

hashar closed subtask T125821: Provide a HHVM package for jessie-wikimedia matching version of trusty-wikimedia as Resolved.May 27 2016, 2:29 PM
hashar mentioned this in T119140: [keyresult] Migrate as many misc CI jobs as possible to Nodepool.Jun 6 2016, 9:40 PM
Paladox added a parent task: T137199: Migrate mwext-testextension* jobs to nodepool.Jun 7 2016, 12:03 PM
Paladox removed a parent task: T137199: Migrate mwext-testextension* jobs to nodepool.Jun 7 2016, 12:10 PM
Paladox added a subtask: T137199: Migrate mwext-testextension* jobs to nodepool.
Paladox removed a project: Patch-For-Review.Jun 7 2016, 12:13 PM
Paladox removed a subscriber: gerritbot.
greg removed a project: releng-201516-q4.Jun 21 2016, 9:35 PM
Andrew closed subtask T133911: Bump quota of Nodepool instances (contintcloud tenant) as Resolved.Jul 27 2016, 3:23 PM
hashar reopened subtask T133911: Bump quota of Nodepool instances (contintcloud tenant) as Open.Aug 14 2016, 9:35 PM
hashar closed subtask T103786: Investigate installing php5.3 on trusty and/or debian instance as Declined.Nov 3 2016, 9:05 PM
hashar closed subtask T133911: Bump quota of Nodepool instances (contintcloud tenant) as Resolved.Nov 29 2016, 3:13 PM
hashar closed subtask T137199: Migrate mwext-testextension* jobs to nodepool as Resolved.Mar 13 2017, 4:19 PM
hashar closed subtask T135001: Migrate PHPUnit MediaWiki core jobs to Nodepool as Resolved.Mar 13 2017, 4:38 PM
hashar added a comment.Mar 13 2017, 4:43 PM

In the interest of cleaning up the augeas stables our project is I am closing this task. Most everything has been migrated now and the last remaining bits are PHP extensions compilation which is tracked by T134381

hashar closed this task as Resolved.Mar 13 2017, 4:43 PM
hashar closed subtask T134381: Migrate PHP extensions building jobs to Nodepool as Declined.Feb 8 2018, 7:58 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL