Page MenuHomePhabricator

Decommission host erbium
Closed, ResolvedPublic

Description

host erbium hit the 5 year mark and is currently blocking the network switch upgrade (using the rack space I need). I confirmed with Alex that it is no longer in use. Server is not in site.pp, does not have production DNS and not being monitored in puppet.

from IRC from alex
akosiaris: cmjohnson1: erbium was an analytics host. udp2log, it was using the netapps for storage of that IIRC. From the task I understand it's a spare host, but it's old enough to warrant just decomissioning it (5 years old last week)

  • - all system services confirmed offline from production use
  • - set all icinga checks to maint mode/disabled while reclaim/decommmission takes place.
  • - remove system from all lvs/pybal active configuration
  • - any service group puppet/hiera/dsh config removed
  • - remove site.pp (replace with role::spare::system if system isn't shut down immediately during this process.)

START NON-INTERRUPPTABLE STEPS

  • - disable puppet on host
  • - remove all remaining puppet references (include role::spare)
  • - power down host
  • - disable switch port
  • - switch port assignment noted on this task (for later removal)
  • - remove production dns entries
  • - puppet node clean, puppet node deactivate

END NON-INTERRUPPTABLE STEPS

  • - system disks wiped (by onsite)
  • - IF DECOM: system unracked and decommissioned (by onsite), update racktables with result
  • - IF DECOM: switch port configration removed from switch once system is unracked.
  • - IF DECOM: mgmt dns entries removed.
  • - IF RECLAIM: system added back to spares tracking (by onsite)

Event Timeline

Change 405012 had a related patch set uploaded (by Cmjohnson; owner: Cmjohnson):
[operations/dns@master] Removing mgmt dns entries for decom host erbium

https://gerrit.wikimedia.org/r/405012

@Framawiki: The one you can already find in the patch.

Change 405012 merged by Cmjohnson:
[operations/dns@master] Removing mgmt dns entries for decom host erbium

https://gerrit.wikimedia.org/r/405012

RobH triaged this task as Medium priority.Feb 16 2018, 5:34 PM
RobH edited projects, added ops-eqiad; removed Patch-For-Review.
Cmjohnson updated the task description. (Show Details)