Page MenuHomePhabricator

Toolforge: move services nodes from eqiad to eqiad1
Closed, ResolvedPublic

Description

We need to move/reallocate services nodes from eqiad to eqiad1.

Care should be taken with packages (aptly repos) and other extra services that were reallocated in the trusty->stretch puppet refactor.

Event Timeline

I can't remember why we decided to duplicate the services. We have new puppet code for stretch and for the new grid, which means the new servers aren't grid submit hosts.
I believe we can just build a couple of new -services nodes in eqiad1, rescue all the binary packages, and shutdown the old -services nodes once we don't need them anymore.

I can't remember why we decided to duplicate the services. We have new puppet code for stretch and for the new grid, which means the new servers aren't grid submit hosts.
I believe we can just build a couple of new -services nodes in eqiad1, rescue all the binary packages, and shutdown the old -services nodes once we don't need them anymore.

Correction, services nodes run the updatetools service, which interacts with the grid.

Mentioned in SAL (#wikimedia-cloud) [2019-01-14T13:47:01Z] <arturo> T213421 create tools-services-03 and tools-services-04 (stretch) they will use the new puppet role role::wmcs::toolforge::services

Mentioned in SAL (#wikimedia-cloud) [2019-01-14T13:53:19Z] <arturo> T213421 delete tools-services-03/04 and create them with another prefix: tools-sge-services-03/04 to actually use the new role

Mentioned in SAL (#wikimedia-cloud) [2019-01-14T14:00:26Z] <arturo> T213421 disable updatetools in the new services nodes while building them

Change 484233 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] toolforge: aptly: create stretch/jessie repos

https://gerrit.wikimedia.org/r/484233

Change 484233 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] toolforge: aptly: create stretch/jessie repos

https://gerrit.wikimedia.org/r/484233

Change 485030 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] aptly: add required ferm rule

https://gerrit.wikimedia.org/r/485030

Change 485030 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] aptly: add required ferm rule

https://gerrit.wikimedia.org/r/485030

Change 485035 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] toolforge: use new instance for aptly server

https://gerrit.wikimedia.org/r/485035

Change 485035 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] toolforge: use new instance for aptly server

https://gerrit.wikimedia.org/r/485035

Mentioned in SAL (#wikimedia-cloud) [2019-01-17T17:16:50Z] <arturo> T213421 shutdown tools-services-01/02. Will delete VMs after a grace period

TODO:

Mentioned in SAL (#wikimedia-cloud) [2019-01-24T11:09:32Z] <arturo> T213421 delete tools-services-01/02

Instances deleted and documentation updated.

Change 486270 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] toolforge: cleanup unused code for services nodes

https://gerrit.wikimedia.org/r/486270

Change 486270 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] toolforge: cleanup unused code for services nodes

https://gerrit.wikimedia.org/r/486270