Page MenuHomePhabricator

Migrate CI labs slaves to use /srv instead of /mnt
Closed, ResolvedPublic

Description

/mnt is legacy. Time to move to /srv :]

Details

Related Gerrit Patches:
operations/puppet : productionMigrate puppet compiler instance from /mnt to /srv
operations/puppet : productioncontint: move from /mnt to /srv
operations/puppet : productioncontint: move to ::srv role from ::mnt
operations/puppet : productioncontint: unmount /mnt/home/jenkins-deploy/tmpfs
operations/puppet : productioncontint: migrate package_builder from /mnt to /srv
operations/puppet : productioncontint: create /srv based directory hierarchy
operations/puppet : productioncontint: migrate browsertest redis to /srv
operations/puppet : productioncontint: add a tmpfs on /srv
operations/puppet : productioncontint: mount /srv the same as /mnt
operations/puppet : productioncontint: migrate castor server to /srv
operations/puppet : productioncontint: remove obsolete files { ensure => absent }
integration/jenkins : masterChange tmpfs to /srv instead of /mnt
integration/config : masterMigrate castor-save to use /srv instead of /mnt

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 22 2016, 3:06 PM

Change 312262 had a related patch set uploaded (by Hashar):
contint: mount /srv the same as /mnt

https://gerrit.wikimedia.org/r/312262

Change 312267 had a related patch set uploaded (by Hashar):
contint: migrate browsertest redis to /srv

https://gerrit.wikimedia.org/r/312267

Change 312270 had a related patch set uploaded (by Hashar):
contint: migrate package_builder from /mnt to /srv

https://gerrit.wikimedia.org/r/312270

Change 312275 had a related patch set uploaded (by Hashar):
contint: remove obsoletes file { ensure => absent }

https://gerrit.wikimedia.org/r/312275

Change 312313 had a related patch set uploaded (by Hashar):
contint: create /srv based directory hierarchy

https://gerrit.wikimedia.org/r/312313

Change 312323 had a related patch set uploaded (by Hashar):
Migrate castor-save to use /srv instead of /mnt

https://gerrit.wikimedia.org/r/312323

Change 312323 merged by jenkins-bot:
Migrate castor-save to use /srv instead of /mnt

https://gerrit.wikimedia.org/r/312323

Change 312322 had a related patch set uploaded (by Hashar):
contint: migrate castor server to /srv

https://gerrit.wikimedia.org/r/312322

Change 312328 had a related patch set uploaded (by Hashar):
contint: add a tmpfs on /srv

https://gerrit.wikimedia.org/r/312328

Change 312330 had a related patch set uploaded (by Hashar):
Change tmpfs to /srv instead of /mnt

https://gerrit.wikimedia.org/r/312330

I am 80% done with it.

Then make the slave::common::labs role to use role::labs::lvm::srv instead of the mnt one. But, the puppet compiler does not use the CI puppet master and is stuck to /mnt for now :(

There are impact on the beta cluster slave as well. But it has a conflicting cherry pick so it will not break magically. Should be easily fixable.

Change 312330 merged by jenkins-bot:
Change tmpfs to /srv instead of /mnt

https://gerrit.wikimedia.org/r/312330

Change 312518 had a related patch set uploaded (by Hashar):
contint: unmount /mnt/home/jenkins-deploy/tmpfs

https://gerrit.wikimedia.org/r/312518

Mentioned in SAL (#wikimedia-releng) [2016-09-23T14:04:51Z] <hashar> remove the /mnt based tmpfs for T146381 / https://gerrit.wikimedia.org/r/#/c/312518/ via: salt -v '*' cmd.run 'umount /mnt/home/jenkins-deploy/tmpfs'

Change 312519 had a related patch set uploaded (by Hashar):
contint: move to ::srv role from ::mnt

https://gerrit.wikimedia.org/r/312519

Change 312523 had a related patch set uploaded (by Hashar):
contint: move from /mnt to /srv

https://gerrit.wikimedia.org/r/312523

Change 312275 abandoned by Hashar:
contint: remove obsolete files { ensure => absent }

Reason:
Squashed in https://gerrit.wikimedia.org/r/#/c/312523/

https://gerrit.wikimedia.org/r/312275

Change 312322 abandoned by Hashar:
contint: migrate castor server to /srv

Reason:
Squashed in https://gerrit.wikimedia.org/r/#/c/312523/

https://gerrit.wikimedia.org/r/312322

Change 312262 abandoned by Hashar:
contint: mount /srv the same as /mnt

Reason:
Squashed in https://gerrit.wikimedia.org/r/#/c/312523/

https://gerrit.wikimedia.org/r/312262

Change 312328 abandoned by Hashar:
contint: add a tmpfs on /srv

Reason:
Squashed in https://gerrit.wikimedia.org/r/#/c/312523/

https://gerrit.wikimedia.org/r/312328

Change 312267 abandoned by Hashar:
contint: migrate browsertest redis to /srv

Reason:
Squashed in https://gerrit.wikimedia.org/r/#/c/312523/

https://gerrit.wikimedia.org/r/312267

Change 312313 abandoned by Hashar:
contint: create /srv based directory hierarchy

Reason:
Squashed in https://gerrit.wikimedia.org/r/#/c/312523/

https://gerrit.wikimedia.org/r/312313

Change 312270 abandoned by Hashar:
contint: migrate package_builder from /mnt to /srv

Reason:
Squashed in https://gerrit.wikimedia.org/r/#/c/312523/

https://gerrit.wikimedia.org/r/312270

Change 312518 abandoned by Hashar:
contint: unmount /mnt/home/jenkins-deploy/tmpfs

Reason:
Squashed in https://gerrit.wikimedia.org/r/#/c/312523/

https://gerrit.wikimedia.org/r/312518

Change 312519 abandoned by Hashar:
contint: move to ::srv role from ::mnt

Reason:
Squashed in https://gerrit.wikimedia.org/r/#/c/312523/

https://gerrit.wikimedia.org/r/312519

hashar triaged this task as Medium priority.Sep 26 2016, 11:06 AM

This is mostly done. Both deployment-prep and CI have the patch cherry picked.

Have to migrate the puppet-compiler instance.

Change 330412 had a related patch set uploaded (by Hashar):
Migrate puppet compiler instance from /mnt to /srv

https://gerrit.wikimedia.org/r/330412

All CI slaves attached to standalone puppet masters have been migrated. The last one to move is the puppet compiler.

I have rebased the couple pending patches in Gerrit. Poked Giuseppe about it on https://gerrit.wikimedia.org/r/#/c/330412/ so we can find a time to migrate the last instance and merge both patches.

hashar added a subscriber: Joe.Jul 18 2017, 9:45 AM

I have quickly talked about it with @Joe . The migration plan is:

compiler02.puppet3-diffs.eqiad.wmflabs is attached to the global labs puppet master, hence the patch needs to be merged first:

https://gerrit.wikimedia.org/r/#/c/312523/move from /mnt to /srv
https://gerrit.wikimedia.org/r/#/c/330412/Migrate puppet compiler instance from /mnt to /srv

Then on the instance:

  • kill anything that might be using /mnt (lsof /mnt)
  • run puppet
  • make sure /mnt is unmounted and no more in /etc/fstab

Mark Node online again:

https://integration.wikimedia.org/ci/computer/compiler02.puppet3-diffs.eqiad.wmflabs/

Run a build and verify the build and PPC config are in a sane state.

Change 312523 merged by Dzahn:
[operations/puppet@production] contint: move from /mnt to /srv

https://gerrit.wikimedia.org/r/312523

Change 330412 merged by Dzahn:
[operations/puppet@production] Migrate puppet compiler instance from /mnt to /srv

https://gerrit.wikimedia.org/r/330412

hashar closed this task as Resolved.Oct 11 2017, 7:24 PM
hashar added a subscriber: Dzahn.

Done! Thank you @Dzahn for the puppet merges.