Page MenuHomePhabricator
Create Task
Maniphest T176931

Decommission db1035
Closed, ResolvedPublic
Actions

Description

db1035 is alerting low on disk space.
db1035's data has been copied over db1103 so this host is ready to be decommissioned.

START NON-INTERRUPPTABLE STEPS

  • - disable puppet on host
  • - remove all remaining puppet references (include role::spare)
  • - power down host
  • - disable switch port & change switch port label to asset tag
  • - remove production dns entries & remove hostname entries in mgmt dns
  • - puppet node clean, puppet node deactivate, salt key removed

END NON-INTERRUPPTABLE STEPS

  • - system disks wiped (by onsite)
  • - remove hostname label, remove hostname from visible label field in racktables (by onsite)
  • - system added back to decom rack (by onsite)

Details

SubjectRepoBranchLines +/-
s3.hosts: Remove db1035operations/softwaremaster+0 -1
site.php: Add db1035 to spareoperations/puppetproduction+5 -0
mariadb: Remove db1035 from config filesoperations/puppetproduction+1 -11
db-eqiad,db-codfw.php: Remove db1035operations/mediawiki-configmaster+0 -3
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedNoneT134476 Decommission old coredb machines (<=db1050)
 Resolved• CmjohnsonT176931 Decommission db1035

Event Timeline

Marostegui created this task.Sep 28 2017, 5:57 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 28 2017, 5:57 AM
Marostegui triaged this task as Medium priority.Sep 28 2017, 5:57 AM
Marostegui moved this task from Triage to In progress on the DBA board.
Marostegui updated the task description. (Show Details)Sep 28 2017, 6:00 AM
Marostegui mentioned this in T148078: Decommission db1015, db1035, db1044 and db1038.Sep 28 2017, 6:18 AM
gerritbot subscribed.Sep 28 2017, 1:45 PM

Change 381216 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad,db-codfw.php: Remove db1035

https://gerrit.wikimedia.org/r/381216

gerritbot added a project: Patch-For-Review.Sep 28 2017, 1:45 PM
gerritbot added a comment.Sep 28 2017, 1:54 PM

Change 381216 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad,db-codfw.php: Remove db1035

https://gerrit.wikimedia.org/r/381216

Stashbot subscribed.Sep 28 2017, 1:55 PM

Mentioned in SAL (#wikimedia-operations) [2017-09-28T13:55:59Z] <marostegui@tin> Synchronized wmf-config/db-codfw.php: Remove db1035 from config as it will be decommissioned - T176931 (duration: 00m 50s)

Stashbot added a comment.Sep 28 2017, 1:56 PM

Mentioned in SAL (#wikimedia-operations) [2017-09-28T13:56:54Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Remove db1035 from config as it will be decommissioned - T176931 (duration: 00m 49s)

gerritbot added a comment.Sep 28 2017, 1:57 PM

Change 381218 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] mariadb: Remove db1035 from config files

https://gerrit.wikimedia.org/r/381218

gerritbot added a comment.Sep 28 2017, 2:31 PM

Change 381218 merged by Marostegui:
[operations/puppet@production] mariadb: Remove db1035 from config files

https://gerrit.wikimedia.org/r/381218

gerritbot added a comment.Sep 28 2017, 2:34 PM

Change 381224 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] site.php: Add db1035 to spare

https://gerrit.wikimedia.org/r/381224

gerritbot added a comment.Sep 28 2017, 2:36 PM

Change 381224 merged by Marostegui:
[operations/puppet@production] site.php: Add db1035 to spare

https://gerrit.wikimedia.org/r/381224

gerritbot added a comment.Sep 28 2017, 2:37 PM

Change 381225 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/software@master] s3.hosts: Remove db1035

https://gerrit.wikimedia.org/r/381225

Marostegui updated the task description. (Show Details)Sep 28 2017, 2:39 PM
Stashbot added a comment.Sep 28 2017, 2:42 PM

Mentioned in SAL (#wikimedia-operations) [2017-09-28T14:42:12Z] <marostegui> Remove db1035 from tendril - T176931

gerritbot added a comment.Sep 28 2017, 2:43 PM

Change 381225 merged by jenkins-bot:
[operations/software@master] s3.hosts: Remove db1035

https://gerrit.wikimedia.org/r/381225

Stashbot added a comment.Sep 28 2017, 2:51 PM

Mentioned in SAL (#wikimedia-operations) [2017-09-28T14:51:20Z] <marostegui> Stop MySQL on db1035 as server is going to be decommissioned - T176931

Marostegui reassigned this task from Marostegui to • Cmjohnson.Sep 28 2017, 2:56 PM
Marostegui moved this task from In progress to Done on the DBA board.
Marostegui edited projects, added ops-eqiad; removed Patch-For-Review.
Marostegui added a subscriber: • Cmjohnson.

db1035 is ready to be decommissioned by @Cmjohnson
All the configuration references have been removed, it has been set to spare, alerts disabled and MySQL is stopped.

Restricted Application added a project: SRE. · View Herald TranscriptSep 28 2017, 2:56 PM
• Cmjohnson moved this task from Backlog to High Priority Task on the ops-eqiad board.Oct 2 2017, 4:36 PM
Marostegui added a parent task: T134476: Decommission old coredb machines (<=db1050).Oct 11 2017, 8:19 AM
Marostegui added a project: hardware-requests.Nov 7 2017, 9:16 AM
• Cmjohnson updated the task description. (Show Details)Nov 8 2017, 6:49 PM
• Cmjohnson updated the task description. (Show Details)Nov 8 2017, 8:36 PM
• Cmjohnson closed this task as Resolved.Nov 14 2017, 4:23 PM

wiped, racktables updated.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL