Page MenuHomePhabricator

Rack and set up 16 db's db1079-1094
Closed, ResolvedPublic

Description

All 16 servers arrived and have been racked. I evenly distributed them across 4 rows.
4 in A2
4 in B1 (@jcrespo I need you to decom 1 database in row B1....i need the switch port, it's only a 24P switch)
2 in C2
2 in C3
4 in D2

  • - receive in normally
  • - rack
  • - update racktables
  • - label and asset tag servers
  • - cabling
  • - setup bios/idrac/ilom
  • - Raid Setup
  • - add mgmt dns entries for both asset tag and hostname
  • - add production dns entries
  • - setup network ports (description, enable, vlan)
  • - update install_server module
  • - install OS
  • puppet/salt keys signed/accepted

Event Timeline

Restricted Application added subscribers: Zppix, Southparkfan, Aklapper. · View Herald TranscriptMay 13 2016, 6:32 PM

Change 288882 had a related patch set uploaded (by Jcrespo):
Take db1027 out of production needed for unracking

https://gerrit.wikimedia.org/r/288882

Change 288882 merged by Jcrespo:
Take db1027 out of production needed for unracking

https://gerrit.wikimedia.org/r/288882

Change 288943 had a related patch set uploaded (by Jcrespo):
Remove (almost) all references to db1027 on production puppet

https://gerrit.wikimedia.org/r/288943

Change 288945 had a related patch set uploaded (by Jcrespo):
Remove all mentions to db1027, db2008 and db2009 from mediawiki

https://gerrit.wikimedia.org/r/288945

Change 288945 merged by Jcrespo:
Remove all mentions to db1027, db2008 and db2009 from mediawiki

https://gerrit.wikimedia.org/r/288945

Change 288943 merged by Jcrespo:
Remove (almost) all references to db1027 on production puppet

https://gerrit.wikimedia.org/r/288943

Change 289168 had a related patch set uploaded (by Jcrespo):
Remove db1027 from internal dns entries

https://gerrit.wikimedia.org/r/289168

@Cmjohnson db1027 has been removed from puppet (sites, dhcp -not netboot, as usual, a range is used-, salt, puppet cert). DNS is above pending, as usual, waiting for you ok.

db1027 is currently down and ready to be disconnected/unracked.

@jcrespo Do you know the raid configuration you want for each of these servers? I am assuming it's H/W raid. I will update during the ILO setup

@Volans @jcrespo

Slight change in racking for row B
db1085 and db1086 will be in B3 now.

Cmjohnson updated the task description. (Show Details)May 19 2016, 4:26 PM
Cmjohnson updated the task description. (Show Details)May 25 2016, 2:10 PM
Cmjohnson updated the task description. (Show Details)
Cmjohnson updated the task description. (Show Details)May 27 2016, 5:51 PM

Change 291948 had a related patch set uploaded (by Jcrespo):
Update autoinstall to include db1079-94

https://gerrit.wikimedia.org/r/291948

Change 291948 merged by Jcrespo:
Update autoinstall to include db1079-94

https://gerrit.wikimedia.org/r/291948

jcrespo claimed this task.May 31 2016, 4:51 PM
jcrespo updated the task description. (Show Details)
jcrespo changed the task status from Open to Stalled.May 31 2016, 6:04 PM

Stalling until T133398 is completed as I've been warned there could be some network issues.

jcrespo changed the task status from Stalled to Open.Jun 1 2016, 11:34 AM
jcrespo reassigned this task from jcrespo to Cmjohnson.

These are the issues I found:

DHCP/network issues
db1082
db1085
db1086

cannot connect to mgmt
db1089
db1094

Change 292159 had a related patch set uploaded (by Jcrespo):
s/wment/wmnet/ for db1080

https://gerrit.wikimedia.org/r/292159

Change 292159 merged by Jcrespo:
s/wment/wmnet/ for db1080

https://gerrit.wikimedia.org/r/292159

Mgmt issues on db1089 and db1094 are fixed and they're installing now.

db1082 cable was disconnected. Re-attached and no issues with installations.
db1085/86 are having issues with vlans. There may be a switch bug.

Having issues removing from vlans...I delete the interfaces from the range and commit and do show vlans and the interfaces remain. They are double listed in labs-instances and private1-b. Cannot remove from either.

with a little help from faidon the vlans were fixed but failed during install.

Chris

jcrespo closed this task as Resolved.Jun 3 2016, 11:08 AM

All installed, work now and boot by default to disk.

Change 289168 merged by Jcrespo:
Remove db1027 from internal dns entries

https://gerrit.wikimedia.org/r/289168

Change 319495 had a related patch set uploaded (by Dzahn):
remove db1027 remnants (reverse lookup)

https://gerrit.wikimedia.org/r/319495

Change 319495 merged by Dzahn:
remove db1027 remnants (reverse lookup)

https://gerrit.wikimedia.org/r/319495