Page MenuHomePhabricator

decom krypton.eqiad.wmnet
Closed, ResolvedPublic

Description

This task will track the decommission-hardware of server krypton.eqiad.wmnet.

The first 5 steps should be completed by the service owner that is returning the server to DC-ops (for reclaim to spare or decommissioning, dependent on server configuration and age.)

krypton.eqiad.wmnet

Steps for service owner:

  • - all system services confirmed offline from production use
  • - set all icinga checks to maint mode/disabled while reclaim/decommmission takes place.
  • - remove system from all lvs/pybal active configuration
  • - any service group puppet/hiera/dsh config removed
  • - remove site.pp, replace with role(spare::system)
  • - run decom script / ensure removed from icinga, debmonitor, puppetboard
  • - shut down ganeti VM
  • - destroy ganeti VM
  • - remove DNS records

Event Timeline

Dzahn created this task.Aug 29 2019, 11:57 AM
Dzahn updated the task description. (Show Details)Aug 29 2019, 12:19 PM
Dzahn updated the task description. (Show Details)
Dzahn removed a subscriber: RobH.
Joe moved this task from Incoming 🐫 to Doing 😎 on the serviceops board.Sep 16 2019, 2:59 PM

Change 537724 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] site: remove krypton.eqiad.wmnet

https://gerrit.wikimedia.org/r/537724

Change 537724 merged by Dzahn:
[operations/puppet@production] site: remove krypton.eqiad.wmnet

https://gerrit.wikimedia.org/r/537724

Change 537733 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] ATS/varnish: remove krypton backend

https://gerrit.wikimedia.org/r/537733

Change 537733 merged by Dzahn:
[operations/puppet@production] ATS/varnish: remove krypton backend

https://gerrit.wikimedia.org/r/537733

Change 537737 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/dns@master] remove krypton.eqiad.wmnet

https://gerrit.wikimedia.org/r/537737

Mentioned in SAL (#wikimedia-operations) [2019-09-18T19:24:45Z] <mutante> ganeti1001 - deleting krypton.eqiad.wmnet - decom T231546

cookbooks.sre.hosts.decommission executed by dzahn@cumin1001 for hosts: krypton.eqiad.wmnet

  • krypton.eqiad.wmnet
    • Removed from Puppet master and PuppetDB
    • Downtimed host on Icinga
    • No management interface found (likely a VM)
    • Removed from DebMonitor

Change 537737 merged by Dzahn:
[operations/dns@master] remove krypton.eqiad.wmnet

https://gerrit.wikimedia.org/r/537737

Dzahn closed this task as Resolved.Sep 18 2019, 8:14 PM
Dzahn updated the task description. (Show Details)