Page MenuHomePhabricator
Create Task
Maniphest T182397

Decomission eventlog2001
Closed, ResolvedPublic
Actions

Description

Very old host that has not served any production traffic.

  • - all system services confirmed offline from production use
  • - set all icinga checks to maint mode/disabled while reclaim/decommmission takes place.
  • - remove system from all lvs/pybal active configuration
  • - any service group puppet/hiera/dsh config removed
  • - remove site.pp (replace with role::spare if system isn't shut down immediately during this process.) -> replaced with role::spare::system

START NON-INTERRUPPTABLE STEPS

  • - disable puppet on hosts
  • - remove all remaining puppet references (include role::spare)
  • - power down hosts
  • - disable switch ports
  • - switch port assignment noted on this task (for later removal)
  • - remove production dns entries
  • - puppet node clean, puppet node deactivate, salt key removed

END NON-INTERRUPPTABLE STEPS

  • - system disks wiped (by onsite)
  • - system unracked and decommissioned (by onsite), update racktables with result
  • - switch port configration removed from switch once system is unracked.
  • - mgmt dns entries removed.

Details

SubjectRepoBranchLines +/-
Decomm: Remove production and mgmt DNS entries for eventlog2001operations/dnsmaster+2 -4
Customize query in gerrit

Event Timeline

elukey triaged this task as Medium priority.Dec 8 2017, 10:33 AM
elukey created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 8 2017, 10:33 AM
elukey added a project: Analytics.Dec 8 2017, 10:34 AM
• fdans moved this task from Incoming to Radar on the Analytics board.Dec 11 2017, 4:45 PM
• Cmjohnson assigned this task to Papaul.Dec 12 2017, 2:51 PM
• Cmjohnson lowered the priority of this task from Medium to Low.
• Cmjohnson edited projects, added ops-codfw, DC-Ops; removed ops-eqiad.
• Cmjohnson added subscribers: Papaul, • Cmjohnson.

assigning to @Papaul and correct data center

gerritbot subscribed.Dec 12 2017, 3:21 PM

Change 397842 had a related patch set uploaded (by Cmjohnson; owner: Cmjohnson):
[operations/puppet@production] Removing decom'd server eventlog2001 from site.pp and dhcpd file T182397

https://gerrit.wikimedia.org/r/397842

gerritbot added a project: Patch-For-Review.Dec 12 2017, 3:21 PM
gerritbot added a comment.Dec 12 2017, 3:27 PM

Change 397842 merged by Cmjohnson:
[operations/puppet@production] Removing decom'd server eventlog2001 from site.pp and dhcpd file T182397

https://gerrit.wikimedia.org/r/397842

gerritbot added a comment.Dec 12 2017, 3:39 PM

Change 397848 had a related patch set uploaded (by Cmjohnson; owner: Cmjohnson):
[operations/dns@master] Removing productin DNS only of eventlog2001 mgmt should stay until unracked T182397

https://gerrit.wikimedia.org/r/397848

gerritbot added a comment.Dec 12 2017, 3:40 PM

Change 397848 merged by Cmjohnson:
[operations/dns@master] Removing productin DNS only of eventlog2001 mgmt should stay until unracked T182397

https://gerrit.wikimedia.org/r/397848

• Cmjohnson updated the task description. (Show Details)Dec 12 2017, 3:41 PM

Switch port is ge-5/0/9 labeled eventlog2001-decommed

@Papaul all yours

Papaul updated the task description. (Show Details)Dec 13 2017, 4:03 PM
Papaul updated the task description. (Show Details)Dec 13 2017, 5:30 PM
gerritbot added a comment.Dec 13 2017, 6:31 PM

Change 398090 had a related patch set uploaded (by Papaul; owner: Papaul):
[operations/dns@master] Decomm: Remove production and mgmt DNS entries for eventlog2001

https://gerrit.wikimedia.org/r/398090

gerritbot added a comment.Dec 13 2017, 7:05 PM

Change 398090 merged by RobH:
[operations/dns@master] Decomm: Remove production and mgmt DNS entries for eventlog2001

https://gerrit.wikimedia.org/r/398090

RobH closed this task as Resolved.Dec 13 2017, 7:15 PM
RobH removed Papaul as the assignee of this task.
RobH removed a project: Patch-For-Review.
RobH updated the task description. (Show Details)
MoritzMuehlenhoff reopened this task as Open.Jan 2 2018, 12:39 PM
MoritzMuehlenhoff subscribed.

This host still shows up in puppetdb, i.e. misses the deactivate step (e.g. visible in https://servermon.wikimedia.org/hosts/)

RobH closed this task as Resolved.Feb 14 2018, 11:53 PM
RobH claimed this task.
RobH subscribed.
In T182397#3866939, @MoritzMuehlenhoff wrote:

This host still shows up in puppetdb, i.e. misses the deactivate step (e.g. visible in https://servermon.wikimedia.org/hosts/)

Not showing there now, someone did a cleanup.

MoritzMuehlenhoff added a comment.Feb 15 2018, 11:21 AM
In T182397#3974123, @RobH wrote:

Not showing there now, someone did a cleanup.

Not quite, that is related to some changes in Puppet 4 and their interaction with servermon. Alex configured some mechanism to clean out obsolete systems after (10?) days a while back (and that much time has passed by now for eventlog2001).

Aklapper edited projects, added Analytics-Radar; removed Analytics.Jun 10 2020, 6:44 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL