Page MenuHomePhabricator

Decommission mw1170-mw1179
Closed, ResolvedPublic

Description

  • - all system services confirmed offline from production use
  • - set all icinga checks to maint mode/disabled while reclaim/decommmission takes place.
  • - remove system from all lvs/pybal active configuration
  • - any service group puppet/hiera/dsh config removed
  • - replace with role::spare::system

START NON-INTERRUPPTABLE STEPS

END NON-INTERRUPPTABLE STEPS

  • - system disks wiped (by onsite)
  • - system unracked and decommissioned (by onsite), update racktables with result
  • - switch port configration removed from switch once system is unracked.
  • - mgmt dns entries removed.

Event Timeline

Mentioned in SAL (#wikimedia-operations) [2017-06-19T11:01:15Z] <_joe_> depooling mw1170-mw1179 for decommissioning, T168271

Change 359920 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/puppet@production] role::mediawiki::appservers: move mw1170-1179 to role::spare::system

https://gerrit.wikimedia.org/r/359920

Change 359920 merged by Giuseppe Lavagetto:
[operations/puppet@production] role::mediawiki::appservers: move mw1170-1179 to role::spare::system

https://gerrit.wikimedia.org/r/359920

@Cmjohnson please proceed to decom/derack these servers and rack new ones in their place.

Mentioned in SAL (#wikimedia-operations) [2017-06-26T21:50:12Z] <robh> shutting down and decommissioning mw117[0-9] per T168271

RobH added a subscriber: RobH.

ge-6/0/9 up up mw1170
ge-6/0/10 up up mw1171
ge-6/0/11 up up mw1172
ge-6/0/12 up up mw1173
ge-6/0/13 up up mw1174
ge-6/0/14 up up mw1175
ge-6/0/15 up up mw1176
ge-6/0/16 up up mw1177
ge-6/0/17 up up mw1178
ge-6/0/18 up down mw1179

Change 361581 had a related patch set uploaded (by RobH; owner: RobH):
[operations/puppet@production] decom mw117[0-9]

https://gerrit.wikimedia.org/r/361581

Change 361582 had a related patch set uploaded (by RobH; owner: RobH):
[operations/dns@master] mw117[0-9] decommission

https://gerrit.wikimedia.org/r/361582

Change 361581 merged by RobH:
[operations/puppet@production] decom mw117[0-9]

https://gerrit.wikimedia.org/r/361581

Change 361582 merged by RobH:
[operations/dns@master] mw117[0-9] decommission

https://gerrit.wikimedia.org/r/361582

RobH updated the task description. (Show Details)

All non-onsite steps have been completed, and these hosts now await disk wipes.

Joe raised the priority of this task from Medium to High.Aug 28 2017, 7:52 AM
Cmjohnson updated the task description. (Show Details)

all steps have been completed.