Page MenuHomePhabricator
Create Task
Maniphest T375239

Decommission frack hosts: frlog2001
Closed, ResolvedPublic
Actions

Description

This task will track the decommission-hardware of:

  • frlog2001.frack.codfw.wmnet

With the launch of updates to the decom cookbook, the majority of these steps can be handled by the service owners directly. The DC Ops team only gets involved once the system has been fully removed from service and powered down by the decommission cookbook.

frlog2001.frack.codfw.wmnet

Steps for service owner:

  • - all system services confirmed offline from production use
  • - set all icinga checks to maint mode/disabled while reclaim/decommmission takes place. (likely done by script)
  • - remove system from all lvs/pybal active configuration
  • - any service group puppet/hiera/dsh config removed
  • - netbox update to decommissioning status
  • - host powered down
  • - remove all remaining puppet references and all host entries in the puppet repo
  • - remove from icinga config - https://gerrit.wikimedia.org/r/c/operations/puppet/+/1074538
  • - remove from dns config - https://gerrit.wikimedia.org/r/c/operations/dns/+/1074537
  • - reassign task from service owner to no owner and ensure the site project (ops-sitename depending on site of server) is assigned.

End service owner steps / Begin DC-Ops team steps:

  • - system disks removed (by onsite)
  • - determine system age, under 5 years are reclaimed to spare, over 5 years are decommissioned.
  • - IF DECOM: system unracked and decommissioned (by onsite), update netbox with result and set state to offline
  • - IF DECOM: mgmt dns entries removed.
  • - IF RECLAIM: set netbox state to 'inventory' and hostname to asset tag

Details

Related Changes in Gerrit:
SubjectAuthorRepoBranchLines +/-
icinga: remove frlog2001 and frpm2001 for decomDwisehauptoperations/puppetproduction+0 -26
frack: remove frlog2001 and frpm2001 for decomDwisehauptoperations/dnsmaster+0 -4
Customize query in gerrit

Related Objects

Event Timeline

Dwisehaupt created this task.Sep 19 2024, 9:27 PM
Dwisehaupt updated the task description. (Show Details)Sep 19 2024, 9:32 PM
Dwisehaupt claimed this task.Sep 19 2024, 9:35 PM
Dwisehaupt mentioned this in T372933: Install and configure new host frlog2002.
Dwisehaupt moved this task from Triage to In Progress on the fundraising-tech-ops board.
Dwisehaupt mentioned this in T375151: codfw:frack:servers migration task.Sep 19 2024, 9:50 PM
Dwisehaupt updated the task description. (Show Details)Sep 20 2024, 9:15 PM
gerritbot added a comment.Sep 20 2024, 9:20 PM

Change #1074537 had a related patch set uploaded (by Dwisehaupt; author: Dwisehaupt):

[operations/dns@master] frack: remove frlog2001 and frpm2001 for decom

https://gerrit.wikimedia.org/r/1074537

gerritbot added a project: Patch-For-Review.Sep 20 2024, 9:20 PM
gerritbot added a comment.Sep 20 2024, 9:23 PM

Change #1074538 had a related patch set uploaded (by Dwisehaupt; author: Dwisehaupt):

[operations/puppet@production] icinga: remove frlog2001 and frpm2001 for decom

https://gerrit.wikimedia.org/r/1074538

Dwisehaupt updated the task description. (Show Details)Sep 20 2024, 9:28 PM

Host powered off and downtime set. Patches created for removal from dns and icinga. Ready for the next steps as those changes can happen in the background.

gerritbot added a comment.Sep 20 2024, 9:30 PM

Change #1074537 merged by Dwisehaupt:

[operations/dns@master] frack: remove frlog2001 and frpm2001 for decom

https://gerrit.wikimedia.org/r/1074537

Dwisehaupt removed Dwisehaupt as the assignee of this task.Sep 20 2024, 9:35 PM
Dwisehaupt added a project: ops-codfw.
Dwisehaupt updated the task description. (Show Details)
Restricted Application added a project: DC-Ops. · View Herald TranscriptSep 20 2024, 9:35 PM
Maintenance_bot added a project: SRE.Sep 20 2024, 10:29 PM
Dwisehaupt moved this task from In Progress to Watching on the fundraising-tech-ops board.Sep 20 2024, 10:30 PM
gerritbot added a comment.Sep 23 2024, 8:24 AM

Change #1074538 merged by Filippo Giunchedi:

[operations/puppet@production] icinga: remove frlog2001 and frpm2001 for decom

https://gerrit.wikimedia.org/r/1074538

Maintenance_bot removed a project: Patch-For-Review.Sep 23 2024, 8:30 AM
Jhancock.wm moved this task from Backlog to Decommission on the ops-codfw board.Sep 23 2024, 1:11 PM
Jhancock.wm assigned this task to Papaul.Sep 23 2024, 2:50 PM
Jhancock.wm updated the task description. (Show Details)
Papaul added a comment.Sep 24 2024, 2:45 PM

switch port removed

papaul@fasw-c-codfw# show | compare
[edit interfaces interface-range disabled]
     member "ge-[0-1]/0/12" { ... }
+    member "ge-[0-1]/0/21";
[edit interfaces interface-range vlan-administration]
-    member "ge-[0-1]/0/21";
[edit interfaces]
-   ge-0/0/21 {
-       description frlog2001:eth0;
-   }
-   ge-1/0/21 {
-       description frlog2001:eth1;
-   }
Papaul closed this task as Resolved.Sep 24 2024, 2:45 PM
Jgreen moved this task from Watching to Done on the fundraising-tech-ops board.Oct 18 2024, 7:09 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits