Page MenuHomePhabricator

Decommission db2034
Closed, ResolvedPublic


db2034 is fully ready to be decommissioned. All the service owner steps are done, so pending DCOPs steps.


Decommission Checklist

START NON-INTERRUPPTABLE STEPS - please assign to @RobH for the non-interrupt steps

  • - disable puppet on host
  • - power down host
  • - update status in netbox to decom
  • - disable switch port
  • - switch port assignment noted on this task (for later removal)
  • - remove all remaining puppet references (include role::spare)
  • - remove production dns entries
  • - puppet node clean, puppet node deactivate (handled by wmf-decommission-host)
  • - remove dbmonitor entries on neodymium/sarin: sudo curl -X DELETE https://debmonitor.discovery.wmnet/hosts/${HOST_FQDN} --cert /etc/debmonitor/ssl/cert.pem --key /etc/debmonitor/ssl/server.key (handled by wmf-decommission-host)


  • - system disks wiped (by onsite) use hdparm for ssds and wipe for hdds
  • - IF DECOM: system unracked and decommissioned (by onsite), netbox status to offline
  • - IF DECOM: switch port configration removed from switch once system is unracked. also trace it out since its not labeled on the switch
  • - IF DECOM: add system to decommission tracking google sheet
  • - IF DECOM: mgmt dns entries removed.


Related Gerrit Patches:
operations/dns : masterDNS: Remove mgmt DNS for db2034
operations/dns : masterdb2034 prod dns decom
operations/puppet : productiondecom db2034

Event Timeline

Restricted Application added a project: Operations. · View Herald TranscriptMay 14 2019, 10:03 AM
Marostegui updated the task description. (Show Details)May 21 2019, 5:47 AM
RobH moved this task from Backlog to Decommission on the ops-codfw board.Jun 12 2019, 1:11 PM

cookbooks.sre.hosts.decommission executed by robh@cumin1001 for hosts: db2034.codfw.wmnet

  • db2034.codfw.wmnet
    • Removed from Puppet master and PuppetDB
    • Downtimed host on Icinga
    • Downtimed management interface on Icinga
    • Removed from DebMonitor
RobH added a comment.Aug 23 2019, 11:36 PM

I cannot locate the labeled switch port on the switch, so @Papaul will need to trace and disable this via on-site work.

Change 531986 had a related patch set uploaded (by RobH; owner: RobH):
[operations/puppet@production] decom db2034

Change 531987 had a related patch set uploaded (by RobH; owner: RobH):
[operations/dns@master] db2034 prod dns decom

Change 531986 merged by RobH:
[operations/puppet@production] decom db2034

Change 531987 merged by RobH:
[operations/dns@master] db2034 prod dns decom

RobH reassigned this task from RobH to Papaul.Aug 23 2019, 11:42 PM
RobH removed a project: Patch-For-Review.
RobH updated the task description. (Show Details)

ge-5/0/32 up down db2034

papaul@asw-a-codfw# run show interfaces ge-5/0/32 descriptions    
Interface       Admin Link Description
ge-5/0/32       down  down DISABLED
Papaul updated the task description. (Show Details)Aug 29 2019, 2:52 PM

Change 533247 had a related patch set uploaded (by Papaul; owner: Papaul):
[operations/dns@master] DNS: Remove mgmt DNS for db2034

Change 533247 merged by Papaul:
[operations/dns@master] DNS: Remove mgmt DNS for db2034

Papaul closed this task as Resolved.Aug 29 2019, 3:51 PM
Papaul updated the task description. (Show Details)