Please note this decom CANNOT take place until the new bastion for ulsfo is online.
bast4001.wikimedia.org (WMF5799) is well out of warranty. There were new systems purchased for misc use on T160936, one of those misc systems should replace the bastion role for ulsfo.
Once the new misc system is online in that role, bast4001 should be decommissioned. It (plus a few cp systems already being accounted for) are the last remaining old ulsfo systems. Once they are all offline, we can get rid of the lot of old systems.
bast4001:
- - all system services confirmed offline from production use
- - set all icinga checks to maint mode/disabled while reclaim/decommmission takes place.
- - remove system from all lvs/pybal active configuration
- - any service group puppet/hiera/dsh config removed
- - remove site.pp (replace with role::spare if system isn't shut down immediately during this process.)
START NON-INTERRUPPTABLE STEPS
- - disable puppet on host
- - power down host
- - update status in netbox (inventory for decom, planned for spare)
- - disable switch port
- - switch port assignment noted on this task (for later removal)
- - remove all remaining puppet references (include role::spare)
- - remove production dns entries
- - puppet node clean, puppet node deactivate (handled by wmf-decommission-host)
- - remove dbmonitor entries on neodymium/sarin: sudo curl -X DELETE https://debmonitor.discovery.wmnet/hosts/${HOST_FQDN} --cert /etc/debmonitor/ssl/cert.pem --key /etc/debmonitor/ssl/server.key (handled by wmf-decommission-host)
END NON-INTERRUPPTABLE STEPS
- - system disks wiped (by onsite)
- - unwire and move out of the way in the rack to make space for new hosts to come online.
- - mgmt dns entries removed. (systems are in rack, but with no power/network/mgmt connections, due to there being no storage in ulsfo and the office has no storage for us during the relocation.)
- - switch port config removed
The remainder cannot happen until we are done with ALL the old CP/bastion/lvs systems to unrack them in a batch.