Page MenuHomePhabricator

Decommission db2011
Closed, ResolvedPublic

Description

db2011 has been substituted by db2044- remove it from service and ask dc ops to decommission it.

Decommission Checklist

  • - all system services confirmed offline from production use - should be done by DBA team https://gerrit.wikimedia.org/r/#/c/415271/
  • - set all icinga checks to maint mode/disabled while reclaim/decommmission takes place.
  • - remove system from all lvs/pybal active configuration - should be done by DBA team
  • - any service group puppet/heira/dsh config removed - should be done by DBA team
  • - remove site.pp (replace with role(spare::system) if system isn't shut down immediately during this process.) - should be done by DBA team: https://gerrit.wikimedia.org/r/#/c/413174/

START NON-INTERRUPPTABLE STEPS - please assign to @RobH for the non-interrupt steps

  • - disable puppet on host
  • - power down host
  • - disable switch port
  • - switch port assignment noted on this task (for later removal) asw-a-codfw:ge-6/0/10
  • - remove all remaining puppet references (include role::spare)
  • - remove production dns entries
  • - puppet node clean, puppet node deactivate

END NON-INTERRUPPTABLE STEPS

  • - system disks wiped (by onsite)
  • - IF DECOM: system unracked and decommissioned (by onsite), update racktables with result
  • - IF DECOM: switch port configration removed from switch once system is unracked.
  • - IF DECOM: add system to decommission tracking google sheet
  • - IF DECOM: mgmt dns entries removed.

Event Timeline

jcrespo triaged this task as Normal priority.Feb 21 2018, 2:45 PM
jcrespo created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 21 2018, 2:45 PM

Change 413174 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: set db2011 (old codfw:s2) as spare

https://gerrit.wikimedia.org/r/413174

Change 413174 merged by Jcrespo:
[operations/puppet@production] mariadb: set db2011 (old codfw:s2) as spare

https://gerrit.wikimedia.org/r/413174

Marostegui moved this task from Triage to Next on the DBA board.Feb 21 2018, 4:03 PM
Marostegui moved this task from Next to In progress on the DBA board.

Change 413321 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/software@master] dblists: Updates for db2044 movement

https://gerrit.wikimedia.org/r/413321

Change 413321 merged by Jcrespo:
[operations/software@master] dblists: Updates for db2044 movement

https://gerrit.wikimedia.org/r/413321

Change 414967 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/dns@master] wmnet: Replaced db2011 with db2044

https://gerrit.wikimedia.org/r/414967

Change 414967 merged by Marostegui:
[operations/dns@master] wmnet: Replaced db2011 with db2044

https://gerrit.wikimedia.org/r/414967

Marostegui updated the task description. (Show Details)Feb 28 2018, 12:31 PM
Marostegui added a subscriber: RobH.

Change 415271 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad,db-codfw.php: Remove db2011

https://gerrit.wikimedia.org/r/415271

Marostegui updated the task description. (Show Details)Feb 28 2018, 12:34 PM

Change 415274 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] install_server: Remove db2011

https://gerrit.wikimedia.org/r/415274

Change 415274 merged by Marostegui:
[operations/puppet@production] install_server: Remove db2011

https://gerrit.wikimedia.org/r/415274

Change 415271 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad,db-codfw.php: Remove db2011

https://gerrit.wikimedia.org/r/415271

Mentioned in SAL (#wikimedia-operations) [2018-02-28T12:42:43Z] <marostegui@tin> Synchronized wmf-config/db-codfw.php: Remove db2011 - T187886 (duration: 00m 58s)

Marostegui updated the task description. (Show Details)

Mentioned in SAL (#wikimedia-operations) [2018-02-28T12:44:04Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Remove db2011 - T187886 (duration: 00m 59s)

Marostegui assigned this task to RobH.Feb 28 2018, 12:44 PM
Marostegui moved this task from In progress to Done on the DBA board.

All the DBA steps are done. Assigning it to @RobH so this can continue
Thanks!

RobH moved this task from Backlog to Decommission on the ops-codfw board.Mar 15 2018, 5:13 PM

After the two servers that were decommissioned yesterday. This is the last host to decommission in codfw as part of T176243 \o/

RobH updated the task description. (Show Details)Apr 27 2018, 8:52 PM

Change 429521 had a related patch set uploaded (by RobH; owner: RobH):
[operations/dns@master] decom db2011 prod dns entries

https://gerrit.wikimedia.org/r/429521

Change 429521 merged by RobH:
[operations/dns@master] decom db2011 prod dns entries

https://gerrit.wikimedia.org/r/429521

Change 429522 had a related patch set uploaded (by RobH; owner: RobH):
[operations/puppet@production] decom db2011

https://gerrit.wikimedia.org/r/429522

Change 429522 merged by RobH:
[operations/puppet@production] decom db2011

https://gerrit.wikimedia.org/r/429522

RobH reassigned this task from RobH to Papaul.Apr 27 2018, 8:57 PM
RobH removed a project: Patch-For-Review.
RobH updated the task description. (Show Details)

ready for onsite completion of steps

switch port information

asw-a6-codfw ge-6/0/10

Papaul updated the task description. (Show Details)May 1 2018, 7:15 PM
Papaul updated the task description. (Show Details)

Change 430400 had a related patch set uploaded (by Papaul; owner: Papaul):
[operations/dns@master] DNS: Remove mgmt DNS for db2011

https://gerrit.wikimedia.org/r/430400

Change 430400 merged by Dzahn:
[operations/dns@master] DNS: Remove mgmt DNS for db2011

https://gerrit.wikimedia.org/r/430400

Papaul reassigned this task from Papaul to RobH.May 3 2018, 3:38 PM
Papaul updated the task description. (Show Details)
Papaul added a subscriber: Papaul.

@RobH done on my end

RobH closed this task as Resolved.May 3 2018, 3:42 PM
RobH removed a project: Patch-For-Review.
RobH updated the task description. (Show Details)