Page MenuHomePhabricator

Decommission db1023
Closed, ResolvedPublic

Description

db1023 is ready to be decommissioned:

START NON-INTERRUPPTABLE STEPS

  • - disable puppet on host
  • - remove all remaining puppet references (include role::spare)
  • - power down host
  • - disable switch port & change switch port label to asset tag
  • - remove production dns entries & remove hostname entries in mgmt dns
  • - puppet node clean, puppet node deactivate, salt key removed

END NON-INTERRUPPTABLE STEPS

  • - system disks wiped (by onsite)
  • - remove hostname label, remove hostname from visible label field in racktables (by onsite)
  • - system removed from rack

Details

Related Gerrit Patches:
operations/software : masters6.hosts: Decommission db1023
operations/puppet : productionmariadb: Decommission db1023
operations/mediawiki-config : masterdb-eqiad,db-codfw.php: Decommission db1023

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 29 2017, 7:27 AM

Change 356010 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad,db-codfw.php: Decommission db1023

https://gerrit.wikimedia.org/r/356010

Change 356011 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] mariadb: Decommission db1023

https://gerrit.wikimedia.org/r/356011

Change 356012 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/software@master] s6.hosts: Decommission db1023

https://gerrit.wikimedia.org/r/356012

Marostegui moved this task from Triage to In progress on the DBA board.May 29 2017, 7:37 AM

Change 356010 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad,db-codfw.php: Decommission db1023

https://gerrit.wikimedia.org/r/356010

Mentioned in SAL (#wikimedia-operations) [2017-05-29T08:02:06Z] <marostegui@tin> Synchronized wmf-config/db-codfw.php: Remove db1023 - T166486 (duration: 00m 42s)

Mentioned in SAL (#wikimedia-operations) [2017-05-29T08:02:55Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Remove db1023 - T166486 (duration: 00m 41s)

Change 356011 merged by Marostegui:
[operations/puppet@production] mariadb: Decommission db1023

https://gerrit.wikimedia.org/r/356011

Marostegui updated the task description. (Show Details)

This is ready for @Cmjohnson to take over whenever he can

Change 356012 merged by jenkins-bot:
[operations/software@master] s6.hosts: Decommission db1023

https://gerrit.wikimedia.org/r/356012

Marostegui moved this task from In progress to Done on the DBA board.May 29 2017, 8:36 AM
Cmjohnson moved this task from Backlog to Not urgent on the ops-eqiad board.May 30 2017, 4:30 PM
Cmjohnson moved this task from Not urgent to Decommission on the ops-eqiad board.Jul 20 2017, 3:24 PM
Dzahn added a subscriber: Dzahn.Sep 25 2017, 4:21 PM

@Cmjohnson db1022 and db1023 and their mgmt interfaces are alerting as down in Icinga. Did you shut them down? Looks like they still need to be removed from puppet.

@Dzahn yes they're down... i think these pre-dated the mgmt interface icinga check

Dzahn added a comment.EditedSep 25 2017, 4:25 PM

@Cmjohnson Ah, thanks! It's both, the servers and mgmt though. https://icinga.wikimedia.org/cgi-bin/icinga/status.cgi?host=all&style=hostdetail&hoststatustypes=4&hostprops=2097158

(This ticket is also for db1022 or is that separate)

Cmjohnson updated the task description. (Show Details)Oct 12 2017, 4:49 PM
Cmjohnson closed this task as Resolved.Oct 12 2017, 5:06 PM