Page MenuHomePhabricator

Decommission search-loader1001/2001 VMs
Closed, ResolvedPublic

Description

Creating this ticket to decommission/remove search-loader1001/2001 VMs. See T346039 for more context.

Event Timeline

Change 973880 had a related patch set uploaded (by Bking; author: Bking):

[operations/puppet@production] search-loader: remove references to search-loader[12]001

https://gerrit.wikimedia.org/r/973880

Command should be sudo cookbook sre.hosts.decommission search-loader1001.eqiad.wmnet,search-loader2001.codfw.wmnet -t T351123. I'm at the end of my day, so will run tomorrow.

cookbooks.sre.hosts.decommission executed by bking@cumin2002 for hosts: search-loader2001.codfw.wmnet,search-loader1001.eqiad.wmnet

  • search-loader2001.codfw.wmnet (PASS)
    • Downtimed host on Icinga/Alertmanager
    • Found Ganeti VM
    • VM shutdown
    • Started forced sync of VMs in Ganeti cluster codfw to Netbox
    • Removed from DebMonitor
    • Removed from Puppet master and PuppetDB
    • VM removed
    • Started forced sync of VMs in Ganeti cluster codfw to Netbox
  • search-loader1001.eqiad.wmnet (PASS)
    • Downtimed host on Icinga/Alertmanager
    • Found Ganeti VM
    • VM shutdown
    • Started forced sync of VMs in Ganeti cluster eqiad to Netbox
    • Removed from DebMonitor
    • Removed from Puppet master and PuppetDB
    • VM removed
    • Started forced sync of VMs in Ganeti cluster eqiad to Netbox

Change 973880 merged by Bking:

[operations/puppet@production] search-loader: remove references to search-loader[12]001

https://gerrit.wikimedia.org/r/973880

bking triaged this task as Medium priority.Nov 14 2023, 3:45 PM
bking moved this task from In Progress to Done on the Data-Platform-SRE board.

These VMs have been fully deleted/decommissioned. Closing...