Page MenuHomePhabricator

Decommission db1035
Closed, ResolvedPublic

Description

db1035 is alerting low on disk space.
db1035's data has been copied over db1103 so this host is ready to be decommissioned.

START NON-INTERRUPPTABLE STEPS

  • - disable puppet on host
  • - remove all remaining puppet references (include role::spare)
  • - power down host
  • - disable switch port & change switch port label to asset tag
  • - remove production dns entries & remove hostname entries in mgmt dns
  • - puppet node clean, puppet node deactivate, salt key removed

END NON-INTERRUPPTABLE STEPS

  • - system disks wiped (by onsite)
  • - remove hostname label, remove hostname from visible label field in racktables (by onsite)
  • - system added back to decom rack (by onsite)

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 28 2017, 5:57 AM
Marostegui triaged this task as Medium priority.Sep 28 2017, 5:57 AM
Marostegui moved this task from Triage to In progress on the DBA board.
Marostegui updated the task description. (Show Details)Sep 28 2017, 6:00 AM

Change 381216 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad,db-codfw.php: Remove db1035

https://gerrit.wikimedia.org/r/381216

Change 381216 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad,db-codfw.php: Remove db1035

https://gerrit.wikimedia.org/r/381216

Mentioned in SAL (#wikimedia-operations) [2017-09-28T13:55:59Z] <marostegui@tin> Synchronized wmf-config/db-codfw.php: Remove db1035 from config as it will be decommissioned - T176931 (duration: 00m 50s)

Mentioned in SAL (#wikimedia-operations) [2017-09-28T13:56:54Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Remove db1035 from config as it will be decommissioned - T176931 (duration: 00m 49s)

Change 381218 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] mariadb: Remove db1035 from config files

https://gerrit.wikimedia.org/r/381218

Change 381218 merged by Marostegui:
[operations/puppet@production] mariadb: Remove db1035 from config files

https://gerrit.wikimedia.org/r/381218

Change 381224 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] site.php: Add db1035 to spare

https://gerrit.wikimedia.org/r/381224

Change 381224 merged by Marostegui:
[operations/puppet@production] site.php: Add db1035 to spare

https://gerrit.wikimedia.org/r/381224

Change 381225 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/software@master] s3.hosts: Remove db1035

https://gerrit.wikimedia.org/r/381225

Marostegui updated the task description. (Show Details)Sep 28 2017, 2:39 PM

Mentioned in SAL (#wikimedia-operations) [2017-09-28T14:42:12Z] <marostegui> Remove db1035 from tendril - T176931

Change 381225 merged by jenkins-bot:
[operations/software@master] s3.hosts: Remove db1035

https://gerrit.wikimedia.org/r/381225

Mentioned in SAL (#wikimedia-operations) [2017-09-28T14:51:20Z] <marostegui> Stop MySQL on db1035 as server is going to be decommissioned - T176931

Marostegui moved this task from In progress to Done on the DBA board.
Marostegui edited projects, added ops-eqiad; removed Patch-For-Review.
Marostegui added a subscriber: Cmjohnson.

db1035 is ready to be decommissioned by @Cmjohnson
All the configuration references have been removed, it has been set to spare, alerts disabled and MySQL is stopped.

Restricted Application added a project: Operations. · View Herald TranscriptSep 28 2017, 2:56 PM
Cmjohnson updated the task description. (Show Details)Nov 8 2017, 6:49 PM
Cmjohnson updated the task description. (Show Details)Nov 8 2017, 8:36 PM
Cmjohnson closed this task as Resolved.Nov 14 2017, 4:23 PM

wiped, racktables updated.