Move db1073 to B3
Closed, ResolvedPublic

Description

All the s1 API servers on the same rack (D1), let's move db1073 to B3 row.
D1 is not under any issues now, so there is no need to do this right now (and thus the Normal priority).

Once we are ready to do this, we will ping Chris.

This is probably not required anymore as we will do this: T156226

Will leave it open for now as it is not a bad a idea to have the 3 api host in 3 different racks anyways. We will evaluate tomorrow.

db1072 has been moved out (T156226) to another rack. So right now (well, once db1072 is recloned) we will have 1 of the API servers out from D1.
I still think we should move db1073 out (right now we have 2 apis + the vslow slave in the same rack, D1). Let's do that next week @Cmjohnson.

it is good anyways to spread the API servers across racks.

Let's not do this until db1072 issues are fixed (T156226#2984963)

Hey @Cmjohnson let's move db1073 to B3 on Wednesday if you'd have time?

Change 336612 had a related patch set uploaded (by Marostegui):
db-eqiad.php: Depool db1073

https://gerrit.wikimedia.org/r/336612

Change 336612 merged by jenkins-bot:
db-eqiad.php: Depool db1073

https://gerrit.wikimedia.org/r/336612

Mentioned in SAL (#wikimedia-operations) [2017-02-08T12:43:43Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1073 - T156126 (duration: 00m 40s)

Server has been depooled and downtimed - ready to shut it down whenever @Cmjohnson is at the DC.

Mentioned in SAL (#wikimedia-operations) [2017-02-08T14:54:00Z] <marostegui> Shutdown db1073 for maintenance - https://phabricator.wikimedia.org/T156126

Change 336631 had a related patch set uploaded (by Marostegui):
Update db1072 IP to match rack change to B3

https://gerrit.wikimedia.org/r/336631

Change 336633 had a related patch set uploaded (by Marostegui):
db-codfw,db-eqiad.php: Change db1073 IP

https://gerrit.wikimedia.org/r/336633

Change 336631 merged by Marostegui:
Update db1073 IP to match rack change to B3

https://gerrit.wikimedia.org/r/336631

Change 336633 merged by jenkins-bot:
db-codfw,db-eqiad.php: Change db1073 IP

https://gerrit.wikimedia.org/r/336633

Mentioned in SAL (#wikimedia-operations) [2017-02-08T15:45:47Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: db1073 change IP - T156126 (duration: 00m 40s)

Mentioned in SAL (#wikimedia-operations) [2017-02-08T15:46:37Z] <marostegui@tin> Synchronized wmf-config/db-codfw.php: db1073 change IP - T156126 (duration: 00m 40s)

db1073 has been moved.
DNS updated
db-eqiad,codfw files updated
mysql and replication started finely.
tendril updated

I will pool it in back slowly to once it is warmer and has caught up with the master

thanks @Cmjohnson

Change 336750 had a related patch set uploaded (by Marostegui):
db-eqiad.php: Repool db1073

https://gerrit.wikimedia.org/r/336750

Change 336750 merged by jenkins-bot:
db-eqiad.php: Repool db1073

https://gerrit.wikimedia.org/r/336750

Mentioned in SAL (#wikimedia-operations) [2017-02-09T07:13:01Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1073 - T156126 (duration: 00m 42s)

Marostegui closed this task as Resolved.Feb 9 2017, 7:18 AM

Server is repooled this can be closed.