Page MenuHomePhabricator

Decommission db1041
Closed, ResolvedPublic

Description

db1041 is ready to be decommissioned:

  • - all system services confirmed offline from production use. Removed from mediawiki-config: https://gerrit.wikimedia.org/r/#/c/373276/
  • - set all icinga checks to maint mode/disabled while reclaim/decommmission takes place.
  • - remove system from all lvs/pybal active configuration
  • - any service group puppet/heira/dsh config removed
  • - remove site.pp (replace with role::spare if system isn't shut down immediately during this process.):
  • Host set to spare until @Cmjohnson removes it forever: https://gerrit.wikimedia.org/r/#/c/373282/

START NON-INTERRUPPTABLE STEPS

  • - disable puppet on host
  • - remove all remaining puppet references (include role::spare)
  • - power down host
  • - disable switch port & change switch port label to asset tag
  • - remove production dns entries & remove hostname entries in mgmt dns
  • - puppet node clean, puppet node deactivate, salt key removed

END NON-INTERRUPPTABLE STEPS

  • - system disks wiped (by onsite)
  • - remove hostname label, remove hostname from visible label field in racktables (by onsite)
  • - system added back to decom rack (by onsite)

Related Objects

StatusSubtypeAssignedTask
ResolvedNone
ResolvedCmjohnson

Event Timeline

Marostegui updated the task description. (Show Details)Aug 23 2017, 12:05 PM

Change 373276 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad,db-codfw.php: Remove db1041

https://gerrit.wikimedia.org/r/373276

Change 373276 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad,db-codfw.php: Remove db1041

https://gerrit.wikimedia.org/r/373276

Mentioned in SAL (#wikimedia-operations) [2017-08-23T12:15:41Z] <marostegui@tin> Synchronized wmf-config/db-codfw.php: Remove db1041 to decommission it - T173915 (duration: 00m 48s)

Mentioned in SAL (#wikimedia-operations) [2017-08-23T12:16:50Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Remove db1041 to decommission it - T173915 (duration: 00m 48s)

Marostegui updated the task description. (Show Details)Aug 23 2017, 12:17 PM

Change 373282 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] mariadb: Remove db1041

https://gerrit.wikimedia.org/r/373282

Change 373282 merged by Marostegui:
[operations/puppet@production] mariadb: Remove db1041

https://gerrit.wikimedia.org/r/373282

Mentioned in SAL (#wikimedia-operations) [2017-08-23T13:31:17Z] <marostegui> Stop MySQL on db1041 to get it ready for decommission - T173915

Marostegui updated the task description. (Show Details)Aug 23 2017, 1:32 PM
Cmjohnson moved this task from Backlog to Decommission on the ops-eqiad board.Aug 23 2017, 3:18 PM

Change 373434 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/software@master] s7.hosts: Remove db1041

https://gerrit.wikimedia.org/r/373434

Change 373434 merged by jenkins-bot:
[operations/software@master] s7.hosts: Remove db1041

https://gerrit.wikimedia.org/r/373434

Marostegui moved this task from Triage to Done on the DBA board.

This host is now ready to be decommissioned and ready for @Cmjohnson do the DC-Ops part

Change 373534 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] mariadb: Remove yaml files from db1015 and db1041

https://gerrit.wikimedia.org/r/373534

Change 373534 merged by Marostegui:
[operations/puppet@production] mariadb: Remove yaml files from db1015 and db1041

https://gerrit.wikimedia.org/r/373534

Cmjohnson updated the task description. (Show Details)Nov 8 2017, 8:37 PM
Cmjohnson closed this task as Resolved.Nov 14 2017, 4:25 PM

Wiped, racktables updated