Page MenuHomePhabricator
Create Task
Maniphest T158020

decom carbon
Closed, ResolvedPublic
Actions

Description

carbon.wikimedia.org has been replaced by install1002/2002 and can be decom'ed after Friday, Feb 17th 2017

decommission steps

  • - all system services confirmed offline from production use
  • - set all icinga checks to maint mode/disabled while reclaim/decommmission takes place.
  • - remove system from all lvs/pybal active configuration
  • - any service group puppet/hiera/dsh config removed
  • - remove site.pp (replace with role::spare if system isn't shut down immediately during this process.)

START NON-INTERRUPPTABLE STEPS

  • - disable puppet on host
  • - remove all remaining puppet references (include role::spare)
  • - power down host
  • - disable switch port
  • - remove production dns entries
  • - puppet node clean, puppet node deactivate, salt key removed

END NON-INTERRUPPTABLE STEPS

  • - system disks wiped by onsite
  • - system unracked and decommissioned (by onsite), update racktables with result
  • - switch port configration removed from asw-a-eqiad for scandium when it is unracked.

Details

SubjectRepoBranchLines +/-
remove carbon's production IPsoperations/dnsmaster+0 -5
install: set install1002 as primary install againoperations/puppetproduction+1 -1
install: remove carbon from puppetoperations/puppetproduction+0 -9
install: allow rsync of /home from carbon to install1002operations/puppetproduction+8 -1
install: remove roles from carbon, demote to spareoperations/puppetproduction+3 -10
Customize query in gerrit

Related Objects
Search...

Event Timeline

Dzahn created this task.Feb 14 2017, 12:16 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 14 2017, 12:16 AM
Dzahn updated the task description. (Show Details)Feb 14 2017, 12:21 AM
Dzahn updated the task description. (Show Details)Feb 14 2017, 12:26 AM
gerritbot subscribed.Feb 14 2017, 12:35 AM

Change 337197 had a related patch set uploaded (by Dzahn):
install: remove roles from carbon, demote to spare

https://gerrit.wikimedia.org/r/337197

gerritbot added a project: Patch-For-Review.Feb 14 2017, 12:35 AM
gerritbot added a comment.Feb 14 2017, 12:43 AM

Change 337197 merged by Dzahn:
install: remove roles from carbon, demote to spare

https://gerrit.wikimedia.org/r/337197

Dzahn updated the task description. (Show Details)Feb 14 2017, 12:51 AM
Dzahn mentioned this in T132757: Split carbon's install/mirror roles, provision install1001.Feb 14 2017, 1:07 AM
gerritbot added a comment.Feb 21 2017, 8:25 PM

Change 339019 had a related patch set uploaded (by Dzahn):
install: allow rsync of /home from carbon to install1002

https://gerrit.wikimedia.org/r/339019

gerritbot added a comment.Feb 21 2017, 8:31 PM

Change 339019 merged by Dzahn:
install: allow rsync of /home from carbon to install1002

https://gerrit.wikimedia.org/r/339019

gerritbot added a comment.Feb 21 2017, 8:31 PM

Change 339021 had a related patch set uploaded (by Dzahn):
install: remove carbon from puppet

https://gerrit.wikimedia.org/r/339021

Stashbot subscribed.Feb 21 2017, 8:36 PM

Mentioned in SAL (#wikimedia-operations) [2017-02-21T20:36:33Z] <mutante> rsyncing /home/ dirs excl. dot files, from carbon to install1002 (T158020)

gerritbot added a comment.Feb 21 2017, 8:39 PM

Change 339023 had a related patch set uploaded (by Dzahn):
install: set install1002 as primary install again

https://gerrit.wikimedia.org/r/339023

Stashbot added a comment.Feb 21 2017, 8:44 PM

Mentioned in SAL (#wikimedia-operations) [2017-02-21T20:44:27Z] <mutante> carbon - backup /root data to install1002:/root/root-carbon/ before shutdown (T158020)

gerritbot added a comment.Feb 21 2017, 8:50 PM

Change 339021 merged by Dzahn:
install: remove carbon from puppet

https://gerrit.wikimedia.org/r/339021

Dzahn updated the task description. (Show Details)Feb 21 2017, 8:52 PM
gerritbot added a comment.Feb 21 2017, 9:00 PM

Change 339023 merged by Dzahn:
install: set install1002 as primary install again

https://gerrit.wikimedia.org/r/339023

Stashbot added a comment.Feb 21 2017, 9:31 PM

Mentioned in SAL (#wikimedia-operations) [2017-02-21T21:31:26Z] <mutante> carbon - puppet node clean, node deactivate (T158020)

Dzahn updated the task description. (Show Details)Feb 21 2017, 9:44 PM
gerritbot added a comment.Feb 21 2017, 9:59 PM

Change 339063 had a related patch set uploaded (by Dzahn):
remove carbon's production IPs

https://gerrit.wikimedia.org/r/339063

Stashbot added a comment.Feb 21 2017, 10:01 PM

Mentioned in SAL (#wikimedia-operations) [2017-02-21T22:01:22Z] <mutante> carbon - removed from icinga, shutdown -h now (T158020)

gerritbot added a comment.Feb 21 2017, 10:03 PM

Change 339063 merged by Dzahn:
remove carbon's production IPs

https://gerrit.wikimedia.org/r/339063

Dzahn updated the task description. (Show Details)Feb 21 2017, 10:06 PM
Dzahn removed a project: Patch-For-Review.
Dzahn reassigned this task from Dzahn to RobH.Feb 21 2017, 10:10 PM
Dzahn added a subscriber: RobH.

Hi @RobH see the check boxes above. could you disable the switch port and then hand over? Thanks!

Dzahn added a project: ops-eqiad.Feb 21 2017, 10:29 PM
Dzahn added a subscriber: • Cmjohnson.
RobH reassigned this task from RobH to Dzahn.Feb 21 2017, 10:48 PM
RobH updated the task description. (Show Details)
This comment was removed by RobH.
RobH reassigned this task from Dzahn to • Cmjohnson.Feb 21 2017, 10:53 PM

So this is now ready for wipe. After discussion with Daniel, we have multiple backup options of this data, but we'll put a last chance date of March 1st.

Chris: Please do not wipe this until March 1st, 2017. If no one objects on this task by then, please wipe and decommission according to remaining checkbox steps.

Thanks!

RobH added a project: hardware-requests.Feb 21 2017, 10:54 PM
RobH moved this task from Backlog to Reclaim (Spares/Decommission) on the hardware-requests board.
• Cmjohnson closed this task as Resolved.Mar 3 2017, 4:38 PM

Removed from rack

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits