Page MenuHomePhabricator
Create Task
Maniphest T94836

Create CI slaves using Debian Jessie
Closed, ResolvedPublic
Actions

Description

I would like us to start pooling some Debian Jessie CI slaves. The intended usage would be to migrate the Jenkins jobs that build Debian packages for us.

Alexandros Kosiaris proposed a patch that creates cow image for all the distributions we use https://gerrit.wikimedia.org/r/#/c/194471/ . So we could tie the debian-glue Jenkins jobs to the Jessie slaves, adjust the debian-glue job template to let us specify the distribution to use and the Jessie instance should be able to build it properly.

Details

SubjectRepoBranchLines +/-
contint: do not install zuul on light slavesoperations/puppetproduction+3 -1
contint: role::ci::slave::labs::lightoperations/puppetproduction+20 -0
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedhasharT103972 Setup jobs for operations/software/conftool
 ResolvedhasharT95545 Migrate all debian-glue jobs to Jessie slaves
 ResolvedhasharT102894 Move labs/tools/extdist tox tests to jessie
 ResolvedhasharT103977 Create tox templates that are bound to jessie
 ResolvedhasharT94836 Create CI slaves using Debian Jessie
 DuplicateNoneT94840 Jessie puppet self instance has puppet erroring out ( Could not find dependency File[/etc/ldap/ldap.conf] )
 ResolvedAndrewT94834 role::puppet::self broken on new labs instances
 ResolvedhasharT94999 Jessie has no install candidate for openjdk-6-jdk
 ResolvedhasharT95000 Update puppet for packages having different names in Jessie
 ResolvedhasharT95003 Use systemd for xvfb service on Debian/Jessie
 ResolvedfaidonT95002 Provide Jessie package to fullfil Mediawiki::Packages requirement
 ResolvedDzahnT102623 Mediawiki font packages: switch to Jessie
 ResolvedfgiunchediT103328 Investigate Ubuntu fork of ttf-indic-fonts and bring it in Jessie
 ResolvedMoritzMuehlenhoffT103325 Investigate usage of ttf-ubuntu-font-family which is not available on Jessie
 ResolvedfaidonT103315 Jessie does not have libmemcached10
 ResolvedfaidonT103322 Jessie does not have libvips15
 ResolvedAndrewT95006 Upload jenkins-debian-glue for Jessie on apt.wikimedia.org
 DeclinedhasharT95008 Build Debian package ruby-jsduck for Jessie
 ResolvedAndrewT102108 New jessie instance can't attach to puppet due to wrong certname
 ResolvedAndrewT102193 Add exception-handling to puppet signer
 ResolvedcorenT75592 Debian Jessie image for Labs
 ResolvedcorenT87309 No init script for idmapd on labs jessie instances
 ResolvedhasharT103600 On Jessie CI slaves, install ruby2.1 instead of ruby1.9.3
 ResolvedhasharT103976 Install etcd on Jessie CI slaves

Event Timeline

hashar created this task.Apr 2 2015, 2:36 PM
hashar raised the priority of this task from to Medium.
hashar updated the task description. (Show Details)
hashar added a project: Continuous-Integration-Infrastructure.
hashar added subscribers: hashar, akosiaris.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 2 2015, 2:36 PM
hashar added a comment.Apr 2 2015, 2:42 PM

Created instance i-00000a3b with image "debian-8.0-jessie" and hostname i-00000a3b.eqiad.wmflabs.

integration-slave-jessie-1001

m1.small (1 CPU, 2GB RAM, 20GB root)

hashar added a subtask: T94834: role::puppet::self broken on new labs instances.Apr 2 2015, 3:01 PM
Andrew closed subtask T94834: role::puppet::self broken on new labs instances as Resolved.Apr 2 2015, 4:41 PM
hashar moved this task from Untriaged to In-progress on the Continuous-Integration-Infrastructure board.Apr 3 2015, 12:32 PM
hashar added a comment.Apr 3 2015, 12:35 PM

After some madness with puppet and OpenStack manager, I got the first Jessie CI slave pooled in Jenkins: https://integration.wikimedia.org/ci/computer/integration-slave-jessie-1001/

It has a single Jenkins label DebianJessie and is not meant yet to run tests.

hashar renamed this task from Create CI slaves using Debian Jessie for debian-glue script to Create CI slaves using Debian Jessie.Apr 3 2015, 1:02 PM
hashar set Security to None.
hashar added a subtask: T95002: Provide Jessie package to fullfil Mediawiki::Packages requirement.Apr 3 2015, 1:23 PM
hashar renamed this task from Create CI slaves using Debian Jessie to Create CI slaves using Debian Jessie (tracking).Apr 7 2015, 12:49 PM
hashar closed subtask T94999: Jessie has no install candidate for openjdk-6-jdk as Resolved.Apr 9 2015, 10:57 AM
hashar closed subtask T95000: Update puppet for packages having different names in Jessie as Resolved.Apr 9 2015, 1:49 PM
hashar added a parent task: T95545: Migrate all debian-glue jobs to Jessie slaves.Apr 9 2015, 2:21 PM
Andrew closed subtask T95006: Upload jenkins-debian-glue for Jessie on apt.wikimedia.org as Resolved.Apr 10 2015, 4:09 PM
hashar mentioned this in T98003: Upgrade prod DNS daemons to gdnsd 2.2.0.May 28 2015, 7:09 PM
faidon subscribed.Jun 3 2015, 1:21 PM
In T98003#1320527, @hashar wrote:

I created a single Jessie slave to report on package/puppet/upstart errors. Tracking is T94836. It is not a priority though :-\

How can we make this a priority? Our fleet is increasingly moving towards a newer distribution and having CI match that is essential to our work. The AuthDNS example (T98003) is a good one: this is one of our most critical services.

hashar raised the priority of this task from Medium to High.Jun 9 2015, 2:27 PM
hashar moved this task from In-progress to Next on the Continuous-Integration-Infrastructure board.
JanZerebecki subscribed.Jun 9 2015, 2:31 PM
hashar added a comment.Jun 10 2015, 8:30 PM

@faidon I have poked Release-Engineering-Team mailing list about it. Seems lot of the tasks are SRE related though.

hashar mentioned this in T102106: Update jenkins-debian-glue packages on Jessie to v0.13.0.Jun 11 2015, 9:13 AM
gerritbot subscribed.Jun 11 2015, 9:29 AM

Change 217466 had a related patch set uploaded (by Hashar):
contint: role::ci::slave::labs::light

https://gerrit.wikimedia.org/r/217466

gerritbot added a project: Patch-For-Review.Jun 11 2015, 9:29 AM
hashar added a subtask: T102108: New jessie instance can't attach to puppet due to wrong certname.Jun 11 2015, 9:38 AM
yuvipanda closed subtask T102108: New jessie instance can't attach to puppet due to wrong certname as Resolved.Jun 11 2015, 9:55 AM
akosiaris closed subtask T95008: Build Debian package ruby-jsduck for Jessie as Resolved.Jun 11 2015, 9:55 AM
gerritbot added a comment.Jun 11 2015, 10:26 AM

Change 217476 had a related patch set uploaded (by Hashar):
contint: do not install zuul on light slaves

https://gerrit.wikimedia.org/r/217476

hashar added a comment.EditedJun 11 2015, 10:38 AM

I have pooled a Jessie slave that is very minimal. https://integration.wikimedia.org/ci/computer/integration-lightslave-jessie-1002/ with labels DebianJessie and contintLabsSlave.

hashar reopened subtask T102108: New jessie instance can't attach to puppet due to wrong certname as Open.Jun 11 2015, 2:00 PM
Andrew closed subtask T102108: New jessie instance can't attach to puppet due to wrong certname as Resolved.Jun 13 2015, 4:47 PM
hashar reopened subtask T102108: New jessie instance can't attach to puppet due to wrong certname as Open.Jun 15 2015, 1:12 PM
hashar closed subtask T102108: New jessie instance can't attach to puppet due to wrong certname as Resolved.Jun 15 2015, 8:21 PM
hashar added a subtask: T75592: Debian Jessie image for Labs.Jun 18 2015, 9:45 PM
hashar merged a task: T86728: Migrate Jenkins slaves from Ubuntu Trusty to Debian Jessie.
hashar mentioned this in T102894: Move labs/tools/extdist tox tests to jessie.
hashar added a parent task: T102894: Move labs/tools/extdist tox tests to jessie.
hashar added a subscriber: Krinkle.
hashar lowered the priority of this task from High to Medium.Jun 22 2015, 12:48 PM
hashar closed subtask T103600: On Jessie CI slaves, install ruby2.1 instead of ruby1.9.3 as Resolved.Jun 23 2015, 9:24 PM
hashar reopened subtask T95008: Build Debian package ruby-jsduck for Jessie as Open.Jun 24 2015, 7:46 PM
hashar added a parent task: T103972: Setup jobs for operations/software/conftool.Jun 26 2015, 10:34 AM
hashar mentioned this in T103972: Setup jobs for operations/software/conftool.
hashar added a parent task: T103977: Create tox templates that are bound to jessie.Jun 30 2015, 10:30 AM
hashar removed a parent task: T103972: Setup jobs for operations/software/conftool.Jul 2 2015, 1:15 PM
gerritbot added a comment.Jul 6 2015, 1:33 PM

Change 217466 merged by Faidon Liambotis:
contint: role::ci::slave::labs::light

https://gerrit.wikimedia.org/r/217466

hashar mentioned this in rOPUP266825ab104c: contint: role::ci::slave::labs::light.Jul 6 2015, 1:33 PM
gerritbot added a comment.Jul 6 2015, 1:34 PM

Change 217476 merged by Faidon Liambotis:
contint: do not install zuul on light slaves

https://gerrit.wikimedia.org/r/217476

hashar mentioned this in rOPUP8ce752fd43c0: contint: do not install zuul on light slaves.Jul 6 2015, 1:34 PM
hashar closed subtask T95008: Build Debian package ruby-jsduck for Jessie as Declined.Jul 20 2015, 9:56 AM
hashar reopened subtask T95008: Build Debian package ruby-jsduck for Jessie as Open.
hashar closed subtask T95008: Build Debian package ruby-jsduck for Jessie as Resolved.Jul 20 2015, 1:25 PM
hashar mentioned this in T106446: CI run complains about missing javac.Jul 22 2015, 9:54 AM
hashar closed subtask T103976: Install etcd on Jessie CI slaves as Resolved.Jul 27 2015, 3:21 PM
hashar mentioned this in T107336: Create jenkins slave instance dedicated to Android runs.Jul 30 2015, 3:58 PM
hashar changed the status of subtask T95008: Build Debian package ruby-jsduck for Jessie from Resolved to Declined.Aug 26 2015, 1:28 PM
hashar changed the task status from Open to Stalled.Oct 6 2015, 12:37 PM
hashar closed subtask T95003: Use systemd for xvfb service on Debian/Jessie as Resolved.Dec 4 2015, 6:45 PM
Danny_B added a project: Tracking-Neverending.May 5 2016, 7:44 PM
faidon closed subtask T95002: Provide Jessie package to fullfil Mediawiki::Packages requirement as Resolved.May 9 2016, 3:08 PM
hashar closed this task as Resolved.May 9 2016, 3:21 PM
hashar claimed this task.

Looks fine. integration-slave-jessie-1001 has been created a while ago and puppet is fine.

hashar added a comment.May 9 2016, 3:44 PM

https://gerrit.wikimedia.org/r/287648 drops the transient role::ci::slave::labs::light which is no more needed.

I have also created a second host integration-slave-jessie-1002 for a total of two executors on m1.medium instances. That is good enough since most jobs are running on Nodepool slaves anyway.

Dzahn mentioned this in rOPUPc1ed731b8b31: contint: drop role::ci::slave::labs::light.Jun 17 2016, 6:10 PM
hashar mentioned this in rOPUP46f8e3cc754d: contint: drop role::ci::slave::labs::light.Jun 17 2016, 6:10 PM
Danny_B moved this task from Tag to Should be Goal instead on the Tracking-Neverending board.Jul 11 2016, 6:42 PM
• Phabricator_maintenance added a project: Goal.Aug 13 2016, 8:41 PM
• Phabricator_maintenance renamed this task from Create CI slaves using Debian Jessie (tracking) to Create CI slaves using Debian Jessie.Aug 13 2016, 9:49 PM
• Phabricator_maintenance removed a project: Tracking-Neverending.
hashar moved this task from Next to Done on the Continuous-Integration-Infrastructure board.Jun 4 2021, 9:52 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits