Page MenuHomePhabricator

Decommission db1064
Closed, ResolvedPublic

Description

db1064 is a slave in x1 that needs to be decommissioned.

db1064

Decommission Checklist

START NON-INTERRUPPTABLE STEPS - please assign to @RobH for the non-interrupt steps

  • - disable puppet on host
  • - power down host
  • - update status in netbox (decommissioning for decom, planned for spare)
  • - disable switch port
  • - switch port assignment noted on this task (for later removal) - asw2-d-eqiad:ge-1/0/11
  • - remove all remaining puppet references (include role::spare)
  • - remove production dns entries
  • - puppet node clean, puppet node deactivate (handled by wmf-decommission-host)
  • - remove dbmonitor entries on neodymium/sarin: sudo curl -X DELETE https://debmonitor.discovery.wmnet/hosts/${HOST_FQDN} --cert /etc/debmonitor/ssl/cert.pem --key /etc/debmonitor/ssl/server.key (handled by wmf-decommission-host)

END NON-INTERRUPPTABLE STEPS

  • - system disks wiped (by onsite) use hdparm for ssds and wipe for hdds
  • - IF DECOM: system unracked and decommissioned (by onsite), update racktables with result
  • - IF DECOM: switch port configration removed from switch once system is unracked.
  • - IF DECOM: add system to decommission tracking google sheet
  • - IF DECOM: mgmt dns entries removed.

Related Objects

Event Timeline

Marostegui moved this task from Triage to Pending comment on the DBA board.

Only tagging DBA for now until our steps are done.

Change 512325 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db1064: Disable notifications

https://gerrit.wikimedia.org/r/512325

Change 512325 merged by Marostegui:
[operations/puppet@production] db1064: Disable notifications

https://gerrit.wikimedia.org/r/512325

Change 512886 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Remove db1064

https://gerrit.wikimedia.org/r/512886

Change 512886 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Remove db1064

https://gerrit.wikimedia.org/r/512886

Mentioned in SAL (#wikimedia-operations) [2019-05-28T19:14:42Z] <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Remove db1064 from config as it will be decommissioned T223217 (duration: 00m 56s)

Mentioned in SAL (#wikimedia-operations) [2019-05-28T19:15:45Z] <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Remove db1064 from config as it will be decommissioned T223217 (duration: 00m 55s)

Change 513958 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db1064: Prepare for decommission

https://gerrit.wikimedia.org/r/513958

Change 513958 merged by Marostegui:
[operations/puppet@production] db1064: Prepare for decommission

https://gerrit.wikimedia.org/r/513958

Mentioned in SAL (#wikimedia-operations) [2019-06-03T08:01:45Z] <marostegui> Remove db1064 from tendril and zarcillo T223217

Marostegui updated the task description. (Show Details)

This host is ready for DCOPs to take over for decommission

cookbooks.sre.hosts.decommission executed by robh@cumin1001 for hosts: db1064.eqiad.wmnet

  • db1064.eqiad.wmnet
    • Removed from Puppet master and PuppetDB
    • Downtimed host on Icinga
    • Downtimed management interface on Icinga
    • Removed from DebMonitor

Change 525637 had a related patch set uploaded (by RobH; owner: RobH):
[operations/dns@master] decom db1064 prod dns

https://gerrit.wikimedia.org/r/525637

Change 525638 had a related patch set uploaded (by RobH; owner: RobH):
[operations/puppet@production] db1064 decom

https://gerrit.wikimedia.org/r/525638

Change 525638 merged by RobH:
[operations/puppet@production] db1064 decom

https://gerrit.wikimedia.org/r/525638

Change 525637 merged by RobH:
[operations/dns@master] decom db1064 prod dns

https://gerrit.wikimedia.org/r/525637

RobH removed RobH as the assignee of this task.Jul 25 2019, 7:58 PM
RobH removed a project: Patch-For-Review.
RobH updated the task description. (Show Details)

Change 548907 had a related patch set uploaded (by Papaul; owner: Papaul):
[operations/dns@master] DNS: Remove mgmt DNS for db1063,db1064,db1065,db1068,db1070 and db1071

https://gerrit.wikimedia.org/r/548907

Change 548907 merged by Papaul:
[operations/dns@master] DNS: Remove mgmt DNS for db1063,db1064,db1065,db1068,db1070 and db1071

https://gerrit.wikimedia.org/r/548907

Papaul updated the task description. (Show Details)
Papaul subscribed.

Complete