Page MenuHomePhabricator

Decommission ocg1001-3
Closed, ResolvedPublic

Description

original task entry

OCG has been decommissioned already as a service, the hosts have been migrated to role::spare::system and all decommission steps up to Steps for DC-OPS (with network switch access) have been completed already.

The servers are out of warranty, so there is no point in returning them to the spares pool.

decom checklist for each system

ocg1001:

  • - all system services confirmed offline from production use
  • - set all icinga checks to maint mode/disabled while reclaim/decommmission takes place.
  • - remove system from all lvs/pybal active configuration
  • - any service group puppet/heira/dsh config removed
  • - remove site.pp (replace with role(spare::system) if system isn't shut down immediately during this process.)

START NON-INTERRUPPTABLE STEPS

END NON-INTERRUPPTABLE STEPS

  • - system disks wiped (by onsite)
  • - IF DECOM: system unracked and decommissioned (by onsite), update racktables with result
  • - IF DECOM: switch port configration removed from switch once system is unracked.
  • - IF DECOM: add system to decommission tracking google sheet
  • - IF DECOM: mgmt dns entries removed.

ocg1002:

  • - all system services confirmed offline from production use
  • - set all icinga checks to maint mode/disabled while reclaim/decommmission takes place.
  • - remove system from all lvs/pybal active configuration
  • - any service group puppet/heira/dsh config removed
  • - remove site.pp (replace with role(spare::system) if system isn't shut down immediately during this process.)

START NON-INTERRUPPTABLE STEPS

END NON-INTERRUPPTABLE STEPS

  • - system disks wiped (by onsite)
  • - IF DECOM: system unracked and decommissioned (by onsite), update racktables with result
  • - IF DECOM: switch port configration removed from switch once system is unracked.
  • - IF DECOM: add system to decommission tracking google sheet
  • - IF DECOM: mgmt dns entries removed.

ocg1003:

  • - all system services confirmed offline from production use
  • - set all icinga checks to maint mode/disabled while reclaim/decommmission takes place.
  • - remove system from all lvs/pybal active configuration
  • - any service group puppet/heira/dsh config removed
  • - remove site.pp (replace with role(spare::system) if system isn't shut down immediately during this process.)

START NON-INTERRUPPTABLE STEPS

END NON-INTERRUPPTABLE STEPS

  • - system disks wiped (by onsite)
  • - IF DECOM: system unracked and decommissioned (by onsite), update racktables with result
  • - IF DECOM: switch port configration removed from switch once system is unracked.
  • - IF DECOM: add system to decommission tracking google sheet
  • - IF DECOM: mgmt dns entries removed.

Event Timeline

Joe created this task.Oct 11 2017, 3:48 PM
Joe removed Joe as the assignee of this task.Oct 11 2017, 3:51 PM
Joe updated the task description. (Show Details)
Cmjohnson moved this task from Backlog to Decommission on the ops-eqiad board.Oct 18 2017, 3:17 PM
RobH claimed this task.Feb 9 2018, 10:30 PM

stealing this, will add in the checklist and manually verify the steps.

@RobH any status update on this?

RobH added a comment.Apr 4 2018, 7:57 PM

Do these need to take priority over other decoms in the backlog?

RobH updated the task description. (Show Details)Apr 4 2018, 7:57 PM
RobH updated the task description. (Show Details)Apr 4 2018, 9:42 PM

Change 424148 had a related patch set uploaded (by RobH; owner: RobH):
[operations/puppet@production] decom ocg100[1-3]

https://gerrit.wikimedia.org/r/424148

Change 424148 merged by RobH:
[operations/puppet@production] decom ocg100[1-3]

https://gerrit.wikimedia.org/r/424148

Change 424149 had a related patch set uploaded (by RobH; owner: RobH):
[operations/dns@master] decom ocg100[1-3] prod dns entries

https://gerrit.wikimedia.org/r/424149

Change 424149 merged by RobH:
[operations/dns@master] decom ocg100[1-3] prod dns entries

https://gerrit.wikimedia.org/r/424149

RobH reassigned this task from RobH to Cmjohnson.Apr 4 2018, 10:19 PM
RobH removed a project: Patch-For-Review.
RobH updated the task description. (Show Details)
RobH added a subscriber: RobH.

Change 451106 had a related patch set uploaded (by Cmjohnson; owner: Cmjohnson):
[operations/dns@master] Removing mgmt dns for decom hosts ocg1001-3

https://gerrit.wikimedia.org/r/451106

Change 451106 merged by Cmjohnson:
[operations/dns@master] Removing mgmt dns for decom hosts ocg1001-3

https://gerrit.wikimedia.org/r/451106

Cmjohnson closed this task as Resolved.Aug 7 2018, 8:26 PM
Cmjohnson updated the task description. (Show Details)