Page MenuHomePhabricator
Create Task
Maniphest T86659

Migrate all jobs to labs slaves
Closed, ResolvedPublic
Actions

Description

The architecture for the disposable VMs (T47499) will have Zuul mergers in the labs infrastructure. None of the production slave (gallium, lanthanum) will be able to reach them due to security constraints.

Thus we need to migrate all jobs tied to productionSlaves label to labs instance (contintLabSlaves).

Tied to gallium

To keep on gallium:

  • integration-docroot-deploy Keep, needs gallium for web server
  • publish-doc Keep, needs gallium for web server

To be moved:

  • jsduck - T86175
    • mediawiki-core-jsduck
    • mediawiki-core-jsduck-publish
    • oojs-core-jsduck
    • oojs-core-jsduck-publish
    • oojs-ui-jsduck
    • oojs-ui-jsduck-publish
    • unicodejs-jsduck
    • unicodejs-jsduck-publish - https://gerrit.wikimedia.org/r/187798
    • VisualEditor-jsduck
    • mwext-GuidedTour-jsduck
    • mwext-GuidedTour-jsduck-publish
    • mwext-MultimediaViewer-jsduck
    • mwext-MultimediaViewer-jsduck-publish
    • mwext-VisualEditor-jsduck
    • mwext-VisualEditor-jsduck-publish

Tied to production slaves (gallium or lanthanum):

  • jobs which have no node: stanza in JJB configuration file. Important

Jobs roaming on any slaves

ssh gallium.wikimedia.org "grep -l '<canRoam>true</canRoam>'  /var/lib/jenkins/jobs/*/config.xml | cut -d\/ -f6"

DONE as of July 3rd 2015.

Search jobs: https://gerrit.wikimedia.org/r/217507

DONE

  • search-extra
  • search-extra-javadoc
  • search-extra-javadoc-publish
  • search-highlighter
  • search-repository-swift

Details

SubjectRepoBranchLines +/-
Convert 'operations-puppet-doc' job to run on a labs slaveintegration/configmaster+28 -5
Migrate *gembuild to Precise labsintegration/configmaster+1 -0
Phase out wikimedia/bugzilla/wikibugsintegration/configmaster+0 -24
Tie search* jobs to labs and Trustyintegration/configmaster+5 -0
Convert 'mediawiki-vagrant-puppet-doc' job to run on a labs slaveintegration/configmaster+9 -6
Pin generic 'phpunit' job to labs slavesintegration/configmaster+1 -1
Use generic phpunit job for operations/mediawiki-configintegration/configmaster+2 -22
Migrate mobile-qunit jobs to new CI slaves in labsintegration/configmaster+1 -2
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedhasharT60772 common gating job for mediawiki core and extensions
 ResolvedhasharT69216 Have unit tests of all wmf deployed extensions pass when installed together, in both PHP-Zend and HHVM (tracking)
 InvalidRyasmeenT90647 Create Jenkins builds for Editing across repositories (MobileFrontend, VisualEditor etc)
 DeclinedNoneT50407 Jenkins: Setup Vagrant for some jobs (tracking)
 DeclinedNoneT45266 Write and implement tests for Wikimedia's Apache configuration (redirects.conf, etc.)
 ResolvedhasharT47499 [EPIC] Run CI jobs in disposable VMs
 ResolvedhasharT70480 Scale merge operations (tracking)
 ResolvedhasharT95046 install/deploy scandium as zuul merger (ci) server
 ResolvedhasharT86661 Disable Gerrit replication to production slaves
 ResolvedKrinkleT91211 gallium and lanthanum disks full (tracking)
 ResolvedhasharT86659 Migrate all jobs to labs slaves
 ResolvedKrinkleT86175 Migrate jsduck jobs to run in labs (and jsduck-publish via integration-publisher)
 ResolvedKrinkleT86174 Migrate JSDuck to Ubuntu Trusty
 ResolvedKrinkleT93143 Migrate .*testextension-zend jobs to labs slaves
 ResolvedhasharT93559 Migrate mediawiki-core-code-coverage job to labs
 ResolvedKrinkleT93558 Add publishing macro for integration.wikimedia.org/cover
 ResolvedhasharT96463 Move integration-zuul-layout* jobs to labs slaves
 ResolvedhasharT95675 Migrate Jenkins job "Global-Dev Dashboard Data" to JJB and Zuul trigger
 ResolvedhasharT98737 Migrate operations-dns-lint Jenkins job to labs instances
 ResolvedhasharT101966 Remove use of label "hasSlaveScripts" in jobs

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
hashar updated the task description. (Show Details)Jan 16 2015, 11:04 AM
hashar added a subtask: T86174: Migrate JSDuck to Ubuntu Trusty.Feb 2 2015, 9:09 PM
hashar updated the task description. (Show Details)
hashar triaged this task as Medium priority.Feb 10 2015, 8:57 PM
Krinkle removed a subtask: T86174: Migrate JSDuck to Ubuntu Trusty.Feb 22 2015, 7:00 PM
Krinkle closed subtask T86175: Migrate jsduck jobs to run in labs (and jsduck-publish via integration-publisher) as Resolved.Mar 3 2015, 11:02 PM
Krinkle renamed this task from Migrate all jobs depending on Zuul git repos out of production slaves to Migrate all jobs to labs slaves.Mar 3 2015, 11:04 PM
Krinkle updated the task description. (Show Details)
Krinkle updated the task description. (Show Details)
Krinkle moved this task from Untriaged to Next on the Continuous-Integration-Infrastructure board.Mar 3 2015, 11:07 PM
gerritbot subscribed.Mar 22 2015, 9:33 PM

Change 198655 had a related patch set uploaded (by Krinkle):
Migrate mobile-qunit jobs to new CI slaves in labs

https://gerrit.wikimedia.org/r/198655

gerritbot added a project: Patch-For-Review.Mar 22 2015, 9:33 PM
gerritbot added a comment.Mar 23 2015, 3:05 AM

Change 198655 merged by jenkins-bot:
Migrate mobile-qunit jobs to new CI slaves in labs

https://gerrit.wikimedia.org/r/198655

Krinkle mentioned this in rCICF6329b4b9f5cc: Migrate mobile-qunit jobs to new CI slaves in labs.Mar 23 2015, 3:56 PM
Krinkle added a parent task: T91211: gallium and lanthanum disks full (tracking).Mar 24 2015, 5:52 AM
Krinkle closed subtask T93143: Migrate .*testextension-zend jobs to labs slaves as Resolved.Mar 25 2015, 1:12 AM
Krinkle removed a project: Patch-For-Review.
Krinkle updated the task description. (Show Details)
Krinkle updated the task description. (Show Details)Mar 25 2015, 1:18 AM
Krinkle closed subtask T93558: Add publishing macro for integration.wikimedia.org/cover as Resolved.Apr 4 2015, 2:56 AM
Krinkle mentioned this in T91211: gallium and lanthanum disks full (tracking).Apr 5 2015, 10:01 AM
hashar updated the task description. (Show Details)Apr 10 2015, 10:22 PM
gerritbot added a comment.Apr 17 2015, 5:23 AM

Change 204706 had a related patch set uploaded (by Legoktm):
Use generic phpunit job for operations/mediawiki-config

https://gerrit.wikimedia.org/r/204706

gerritbot added a project: Patch-For-Review.Apr 17 2015, 5:23 AM
gerritbot added a comment.Apr 17 2015, 5:35 AM

Change 204707 had a related patch set uploaded (by Legoktm):
Pin generic 'phpunit' job to labs slaves

https://gerrit.wikimedia.org/r/204707

gerritbot added a comment.Apr 17 2015, 5:36 AM

Change 204706 merged by jenkins-bot:
Use generic phpunit job for operations/mediawiki-config

https://gerrit.wikimedia.org/r/204706

Legoktm mentioned this in rCICF25494c7ca48b: Use generic phpunit job for operations/mediawiki-config.Apr 17 2015, 5:38 AM
gerritbot added a comment.Apr 17 2015, 5:41 AM

Change 204707 merged by jenkins-bot:
Pin generic 'phpunit' job to labs slaves

https://gerrit.wikimedia.org/r/204707

Legoktm mentioned this in rCICF9f76dcf3e73f: Pin generic 'phpunit' job to labs slaves.Apr 17 2015, 5:41 AM
Legoktm updated the task description. (Show Details)Apr 18 2015, 1:23 AM
gerritbot added a comment.Apr 18 2015, 1:36 AM

Change 204980 had a related patch set uploaded (by Legoktm):
Convert 'mediawiki-vagrant-puppet-doc' job to run on a labs slave

https://gerrit.wikimedia.org/r/204980

gerritbot added a comment.Apr 18 2015, 1:48 AM

Change 204982 had a related patch set uploaded (by Legoktm):
Convert 'operations-puppet-doc' job to run on a labs slave

https://gerrit.wikimedia.org/r/204982

hashar updated the task description. (Show Details)Apr 20 2015, 1:35 PM
hashar closed subtask T96463: Move integration-zuul-layout* jobs to labs slaves as Resolved.Apr 20 2015, 1:55 PM
hashar updated the task description. (Show Details)
Krinkle mentioned this in T96576: https://doc.wikimedia.org/puppet/ does not purge obsolete class documentation.Apr 21 2015, 3:04 PM
gerritbot added a comment.Apr 22 2015, 1:27 PM

Change 204980 merged by jenkins-bot:
Convert 'mediawiki-vagrant-puppet-doc' job to run on a labs slave

https://gerrit.wikimedia.org/r/204980

hashar mentioned this in rCICFac8bdf3a7899: Convert 'mediawiki-vagrant-puppet-doc' job to run on a labs slave.Apr 22 2015, 1:29 PM
hashar updated the task description. (Show Details)Apr 22 2015, 1:35 PM
hashar updated the task description. (Show Details)May 4 2015, 1:33 PM
hashar added subtasks: T95675: Migrate Jenkins job "Global-Dev Dashboard Data" to JJB and Zuul trigger, T97514: Migrate "analytics-*" jobs to Jenkins Job Builder.

I have added some jobs that do not have any node: and hence roam on any of our slaves (the Jenkins XML has: canRoam>true</canRoam>).

hashar added a subtask: T98737: Migrate operations-dns-lint Jenkins job to labs instances.May 19 2015, 9:34 AM
hashar updated the task description. (Show Details)Jun 8 2015, 11:46 AM
hashar mentioned this in T101966: Remove use of label "hasSlaveScripts" in jobs.Jun 10 2015, 11:08 AM
hashar added a subtask: T101966: Remove use of label "hasSlaveScripts" in jobs.
hashar closed subtask T98737: Migrate operations-dns-lint Jenkins job to labs instances as Resolved.Jun 11 2015, 1:13 PM
gerritbot added a comment.Jun 11 2015, 1:29 PM

Change 217507 had a related patch set uploaded (by Hashar):
Tie search* jobs to labs and Trusty

https://gerrit.wikimedia.org/r/217507

hashar updated the task description. (Show Details)Jun 11 2015, 1:31 PM
gerritbot added a comment.Jun 11 2015, 1:34 PM

Change 217507 merged by jenkins-bot:
Tie search* jobs to labs and Trusty

https://gerrit.wikimedia.org/r/217507

hashar mentioned this in rCICFc64d073e55be: Tie search* jobs to labs and Trusty.Jun 11 2015, 1:34 PM
gerritbot added a comment.Jun 11 2015, 1:34 PM

Change 217508 had a related patch set uploaded (by Hashar):
Phase out wikimedia/bugzilla/wikibugs

https://gerrit.wikimedia.org/r/217508

hashar updated the task description. (Show Details)Jun 11 2015, 1:35 PM
gerritbot added a comment.Jun 11 2015, 1:38 PM

Change 217508 merged by jenkins-bot:
Phase out wikimedia/bugzilla/wikibugs

https://gerrit.wikimedia.org/r/217508

hashar mentioned this in rCICF956e7fbf4595: Phase out wikimedia/bugzilla/wikibugs.Jun 11 2015, 1:38 PM
gerritbot added a comment.Jun 11 2015, 1:42 PM

Change 217509 had a related patch set uploaded (by Hashar):
Migrate *gembuild to Precise labs

https://gerrit.wikimedia.org/r/217509

hashar updated the task description. (Show Details)Jun 11 2015, 1:44 PM
gerritbot added a comment.Jun 11 2015, 1:46 PM

Change 217509 merged by jenkins-bot:
Migrate *gembuild to Precise labs

https://gerrit.wikimedia.org/r/217509

hashar mentioned this in rCICF58b76dc64cba: Migrate *gembuild to Precise labs.Jun 11 2015, 1:46 PM
hashar closed subtask T95675: Migrate Jenkins job "Global-Dev Dashboard Data" to JJB and Zuul trigger as Resolved.Jun 25 2015, 12:35 PM
hashar closed subtask T101966: Remove use of label "hasSlaveScripts" in jobs as Resolved.Jul 3 2015, 1:00 PM
hashar updated the task description. (Show Details)
hashar removed a subtask: T97514: Migrate "analytics-*" jobs to Jenkins Job Builder.Jul 3 2015, 1:09 PM
hashar updated the task description. (Show Details)Jul 3 2015, 1:12 PM
hashar updated the task description. (Show Details)Jul 3 2015, 1:24 PM
hashar updated the task description. (Show Details)Jul 3 2015, 1:31 PM
hashar added a comment.Jul 3 2015, 2:20 PM

I believe nothing will run on lanthanum.eqiad.wmnet anymore. What is left to migrate are the jobs currently running on gallium. Some might be migrated to labs, some not.

hashar removed a parent task: T86658: Phase out lanthanum.eqiad.wmnet.Jul 6 2015, 8:40 PM
Krinkle updated the task description. (Show Details)Jul 6 2015, 8:43 PM
hashar closed subtask T93559: Migrate mediawiki-core-code-coverage job to labs as Resolved.Jul 21 2015, 9:37 PM
greg updated the task description. (Show Details)Aug 22 2015, 12:48 AM
hashar lowered the priority of this task from Medium to Low.Sep 14 2015, 8:51 AM
hashar removed a project: Patch-For-Review.

Almost completed. There is still a few jobs on gallium.wikimedia.org though.

gerritbot added a comment.Oct 27 2015, 11:21 AM

Change 204982 merged by jenkins-bot:
Convert 'operations-puppet-doc' job to run on a labs slave

https://gerrit.wikimedia.org/r/204982

hashar updated the task description. (Show Details)Oct 27 2015, 11:24 AM
hashar added a comment.EditedOct 28 2015, 1:59 PM

Status as October 28th 2015

$ ssh gallium.wikimedia.org ls -1 /srv/ssd/jenkins-slave/workspace
integration-docroot-deploy
mediawiki-vagrant-puppet-doc
mwext-VisualEditor-sync-gerrit
publish-on-gallium
Legoktm subscribed.Oct 29 2015, 7:56 AM

I suppose we could turn the integration-docroot-deploy into a rsync job like the doc ones? Is it worth it?

I'm guessing mediawiki-vagrant-puppet-doc is straightforward, similar to the ops puppet one.

Is it even possible to move publish-on-gallium onto labs?

Paladox updated the task description. (Show Details)Feb 29 2016, 3:29 PM
Stashbot mentioned this in T51846: mediawiki/extensions.git does not update some extensions.Sep 8 2016, 10:02 AM
Stashbot subscribed.

Mentioned in SAL [2016-09-08T10:02:05Z] <hashar> Delete mwext-VisualEditor-sync-gerrit job, already got removed by ostriches in 139d17c8f1c4bcf2bb761e13a6501e4d85684066 . The issue in Gerrit (T51846) has been fixed. Poke T86659 , one less job on slaves.

hashar updated the task description. (Show Details)Sep 8 2016, 10:02 AM
Stashbot added a comment.Sep 8 2016, 10:03 AM

Mentioned in SAL [2016-09-08T10:03:29Z] <hashar> Delete Jenkins job https://integration.wikimedia.org/ci/job/mwext-VisualEditor-sync-gerrit/ that has been left behind. It is no more needed. T51846 T86659

Paladox subscribed.Sep 8 2016, 10:11 AM

This is almost finished we just need to do

jobs which have no node: stanza in JJB configuration file. Important

now.

Stashbot added a comment.Nov 24 2016, 8:49 PM

Mentioned in SAL (#wikimedia-releng) [2016-11-24T20:49:59Z] <hashar> make contint1001 Jenkins slave to only builds jobs with a label matching the node https://integration.wikimedia.org/ci/computer/contint1001/configure T86659

hashar closed this task as Resolved.Nov 24 2016, 8:51 PM
hashar claimed this task.

Paladox pointed:

jobs which have no node: stanza in JJB configuration file. Important

I have changed contint1001 slave to:

Only build jobs with label restrictions matching this node

In this mode, Jenkins will only build a project on this node when that project is restricted to certain nodes using a label expression, and that expression matches this node's name and/or labels. This allows a slave to be reserved for certain kinds of jobs. For example, to run performance tests continuously from Jenkins, you can use this setting with # of executors as 1, so that only one performance test runs at any given time, and that one executor won't be blocked by other builds that can be done on other nodes.

This way it only runs jobs that have node: contint1001 :]

The rest of jobs running on the production machine contint1001 are related to publish artifacts for doc.wikimedia.org. They will be moved to another host eventually via another task (cant find it).

hashar moved this task from Next to Done on the Continuous-Integration-Infrastructure board.Jun 4 2021, 9:52 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL