Decommission db1045
Open, NormalPublic

Description

db1045 is ready to be decommissioned after its data was copied to db1099

  • - all system services confirmed offline from production use: Removed from mediawiki-config: https://gerrit.wikimedia.org/r/#/c/375748/
  • - set all icinga checks to maint mode/disabled while reclaim/decommmission takes place.
  • - remove system from all lvs/pybal active configuration
  • - any service group puppet/heira/dsh config removed
  • - remove site.pp (replace with role::spare if system isn't shut down immediately during this process.):
  • Added as spare to site.pp until @Cmjohnson removes it from good once he starts the DC Ops steps: https://gerrit.wikimedia.org/r/#/c/375750/

START NON-INTERRUPPTABLE STEPS

  • - disable puppet on host
  • - remove all remaining puppet references (include role::spare)
  • - power down host
  • - disable switch port & change switch port label to asset tag
  • - remove production dns entries & remove hostname entries in mgmt dns
  • - puppet node clean, puppet node deactivate, salt key removed

END NON-INTERRUPPTABLE STEPS

  • - system disks wiped (by onsite)
  • - remove hostname label, remove hostname from visible label field in racktables (by onsite)
  • - system added back to spares tracking (by onsite)
Restricted Application added a project: Operations. · View Herald TranscriptFri, Sep 1, 4:30 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Marostegui moved this task from Triage to Next on the DBA board.Fri, Sep 1, 4:31 PM
Marostegui triaged this task as Normal priority.
Marostegui updated the task description. (Show Details)Mon, Sep 4, 8:28 AM

Change 375748 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad,db-codfw.php: Remove db1045

https://gerrit.wikimedia.org/r/375748

Change 375750 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] mariadb: Remove db1045 - will be decommissioned

https://gerrit.wikimedia.org/r/375750

Change 375748 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad,db-codfw.php: Remove db1045

https://gerrit.wikimedia.org/r/375748

Mentioned in SAL (#wikimedia-operations) [2017-09-04T08:39:45Z] <marostegui@tin> Synchronized wmf-config/db-codfw.php: Remove db1045 as it will be decommissioned - T174806 (duration: 00m 48s)

Mentioned in SAL (#wikimedia-operations) [2017-09-04T08:40:39Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Remove db1045 as it will be decommissioned - T174806 (duration: 00m 46s)

Change 375750 merged by Marostegui:
[operations/puppet@production] mariadb: Remove db1045 - will be decommissioned

https://gerrit.wikimedia.org/r/375750

Marostegui updated the task description. (Show Details)Mon, Sep 4, 8:47 AM
Marostegui added a subscriber: Cmjohnson.

Mentioned in SAL (#wikimedia-operations) [2017-09-04T08:48:41Z] <marostegui> Stop MySQL on db1045 as it will be decommissioned - T174806

Marostegui moved this task from Next to Done on the DBA board.Mon, Sep 4, 8:50 AM
Marostegui assigned this task to Cmjohnson.

This server is ready to be totally decommissioned and only pending the DC Ops steps, so assigning it to @Cmjohnson for the pending steps

Cmjohnson moved this task from Backlog to Decommission on the ops-eqiad board.Tue, Sep 5, 7:27 PM