Page MenuHomePhabricator

decommission frpm1001.frack.eqiad.wmnet
Closed, ResolvedPublicRequest

Description

This task will track the decommission-hardware of server frpm1001.frack.eqiad.wmnet.

With the launch of updates to the decom cookbook, the majority of these steps can be handled by the service owners directly. The DC Ops team only gets involved once the system has been fully removed from service and powered down by the decommission cookbook.

Steps for service owner:

  • - all system services confirmed offline from production use
  • - remove from icinga monitoring config (nsca_frack.cfg)
  • - migrate archive data
  • - remove all remaining puppet references and all host entries in the puppet repo
  • - remove from firewall policy
  • - mark the host "Decommission" in Netbox
  • - reassign task from service owner to DC ops team member and site project (ops-sitename) depending on site of server

End service owner steps / Begin DC-Ops team steps:

  • - [Note: I don't know whether this works for frack hosts] login to cumin host and run the decom cookbook: cookbook sre.hosts.decommission <host fqdn> -t <phab task>. This does: bootloader wipe, host power down, netbox update to decommissioning status, puppet node clean, puppet node deactivate, debmonitor removal, and run homer.
  • - system disks removed (by onsite)
  • - determine system age, under 5 years are reclaimed to spare, over 5 years are decommissioned.
  • - IF DECOM: system unracked and decommissioned (by onsite), update netbox with result and set state to offline
  • - IF DECOM: mgmt dns entries removed.

Event Timeline

Jgreen updated the task description. (Show Details)
Jgreen updated the task description. (Show Details)

Change 889561 had a related patch set uploaded (by Jgreen; author: Jgreen):

[operations/puppet@production] Remove icinga monitoring of frpm1001.frack.eqiad.wmnet for decom.

https://gerrit.wikimedia.org/r/889561

Change 889561 merged by Jgreen:

[operations/puppet@production] Remove icinga monitoring of frpm1001.frack.eqiad.wmnet for decom.

https://gerrit.wikimedia.org/r/889561

Jgreen added a project: fundraising-tech-ops.
Jgreen moved this task from Triage to In Progress on the fundraising-tech-ops board.
Jgreen updated the task description. (Show Details)
Jgreen edited projects, added ops-eqiad; removed fundraising-tech-ops.
Jgreen added a project: SRE.

Frpm1001 is powered off and all set for decommissioning. Please note, per prior discussion with Willy I did not run the cookbook stuff since this is a frack host.

Change 894044 had a related patch set uploaded (by Jgreen; author: Jgreen):

[operations/dns@master] Add hosts frbast1002,frmon1002,frpig1002, remove frauth1001,frpm1001

https://gerrit.wikimedia.org/r/894044

Change 894044 merged by Jgreen:

[operations/dns@master] Add hosts frbast1002,frmon1002,frpig1002, remove frauth1001,frpm1001

https://gerrit.wikimedia.org/r/894044

Ran offline script, Removed from rack

Jclark-ctr updated the task description. (Show Details)