Page MenuHomePhabricator

Repurpose bast3004 as ganeti node
Closed, DeclinedPublic

Description

The bastions in the caches sites are now running on VMs, as such bast3004 will be repurposed as a Ganeti node (the cluster in esams only had three nodes ATM). This follows the procedure outlined at https://wikitech.wikimedia.org/wiki/Server_Lifecycle#Rename_while_reimaging :

  • Decom bast3004
  • Edit the device page to set the new name (use the hostname, not the FQDN) and set its status from DECOMMISSIONING to PLANNED.
  • Rename the DNS Name of all its IPs (there should be only the management IP at this stage)
  • Delete the interfaces and use the netbox script to provision them again
  • Add VLANs for Ganeti
  • Run the sre.dns.netbox cookbook: DNS/Netbox#Update_generated_records
  • Run the sre.network.configure-switch-interfaces cookbook in order to configure the switch interface
  • Install as ganeti3004
  • Add ganeti3004 to the esams Ganeti cluster
  • Get the physical re-labeling done (open a task for dc-ops)

Event Timeline

MoritzMuehlenhoff updated the task description. (Show Details)

cookbooks.sre.hosts.decommission executed by jmm@cumin2002 for hosts: bast3004.wikimedia.org

  • bast3004.wikimedia.org (WARN)
    • Downtimed host on Icinga/Alertmanager
    • Found physical host
    • Management interface not found on Icinga, unable to downtime it
    • Wiped all swraid, partition-table and filesystem signatures
    • Powered off
    • [Netbox] Set status to Decommissioning, deleted all non-mgmt IPs, updated switch interfaces (disabled, removed vlans, etc)
    • Configured the linked switch interface(s)
    • Removed from DebMonitor
    • Removed from Puppet master and PuppetDB

Change 884882 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Remove bast3004/bast6001

https://gerrit.wikimedia.org/r/884882

Change 884882 merged by Muehlenhoff:

[operations/puppet@production] Remove bast3004/bast6001

https://gerrit.wikimedia.org/r/884882

The server will be decommissioned with the rest old old-esams via T343957