Page MenuHomePhabricator

Decommission db1039
Closed, ResolvedPublic

Description

db1039 is ready to be decommissioned after being finished with data checksums.

  • - all system services confirmed offline from production use: Removed from mediawiki-config: https://gerrit.wikimedia.org/r/#/c/402765/
  • - set all icinga checks to maint mode/disabled while reclaim/decommmission takes place.
  • - remove system from all lvs/pybal active configuration
  • - any service group puppet/heira/dsh config removed
  • - remove site.pp (replace with role::spare if system isn't shut down immediately during this process.):
  • Host set to spare: https://gerrit.wikimedia.org/r/#/c/402766/1

START NON-INTERRUPPTABLE STEPS

  • - disable puppet on host
  • - remove all remaining puppet references (include role::spare)
  • - power down host
  • - disable switch port & change switch port label to asset tag
  • - remove production dns entries & remove hostname entries in mgmt dns
  • - puppet node clean, puppet node deactivate, salt key removed

END NON-INTERRUPPTABLE STEPS

  • - system disks wiped (by onsite)
  • - remove hostname label, remove hostname from visible label field in racktables (by onsite)
  • - system added back to spares tracking (by onsite)

Event Timeline

Marostegui triaged this task as Normal priority.Jan 5 2018, 10:15 AM
Marostegui created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 5 2018, 10:15 AM
Marostegui moved this task from Triage to Next on the DBA board.Jan 5 2018, 10:16 AM

I will start with it on Monday

Marostegui moved this task from Next to In progress on the DBA board.Jan 8 2018, 6:55 AM

Change 402765 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad,db-codfw.php: Remove db1039 from config

https://gerrit.wikimedia.org/r/402765

Change 402766 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] mariadb: Remove db1039

https://gerrit.wikimedia.org/r/402766

Change 402765 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad,db-codfw.php: Remove db1039 from config

https://gerrit.wikimedia.org/r/402765

Mentioned in SAL (#wikimedia-operations) [2018-01-08T07:04:32Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Remove db1039 as it will be decommissioned - T184262 (duration: 00m 50s)

Mentioned in SAL (#wikimedia-operations) [2018-01-08T07:05:29Z] <marostegui@tin> Synchronized wmf-config/db-codfw.php: Remove db1039 as it will be decommissioned - T184262 (duration: 00m 50s)

Change 402766 merged by Marostegui:
[operations/puppet@production] mariadb: Remove db1039

https://gerrit.wikimedia.org/r/402766

Mentioned in SAL (#wikimedia-operations) [2018-01-08T07:17:10Z] <marostegui> Remove db1039 from tendril - T184262

Mentioned in SAL (#wikimedia-operations) [2018-01-08T07:24:13Z] <marostegui> Stop MySQL on db1039 for decommission - T184262

Change 402768 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/software@master] s7.hosts: Remove db1039

https://gerrit.wikimedia.org/r/402768

Marostegui updated the task description. (Show Details)Jan 8 2018, 7:26 AM

Change 402768 merged by jenkins-bot:
[operations/software@master] s7.hosts: Remove db1039

https://gerrit.wikimedia.org/r/402768

Marostegui reassigned this task from Marostegui to Cmjohnson.Jan 8 2018, 7:27 AM
Marostegui moved this task from In progress to Done on the DBA board.
Marostegui added a project: ops-eqiad.
Marostegui added a subscriber: Cmjohnson.

db1039 is now ready to be decommissioned by @Cmjohnson

Restricted Application added a project: Operations. · View Herald TranscriptJan 8 2018, 7:27 AM
Cmjohnson moved this task from Backlog to Decommission on the ops-eqiad board.Jan 9 2018, 3:20 PM
RobH claimed this task.Feb 9 2018, 10:27 PM
RobH moved this task from Backlog to Reclaim (Spares/Decommission) on the hardware-requests board.

Change 428932 had a related patch set uploaded (by Cmjohnson; owner: Cmjohnson):
[operations/puppet@production] Removing db1039 site.pp entry

https://gerrit.wikimedia.org/r/428932

Change 428932 merged by Cmjohnson:
[operations/puppet@production] Removing db1039 site.pp entry

https://gerrit.wikimedia.org/r/428932

Change 429814 had a related patch set uploaded (by Cmjohnson; owner: Cmjohnson):
[operations/dns@master] Removing dns entries for db1039

https://gerrit.wikimedia.org/r/429814

Change 429814 merged by Cmjohnson:
[operations/dns@master] Removing dns entries for db1039

https://gerrit.wikimedia.org/r/429814

Cmjohnson updated the task description. (Show Details)Apr 30 2018, 2:46 PM
Cmjohnson closed this task as Resolved.May 7 2018, 5:13 PM
Cmjohnson updated the task description. (Show Details)