db1019: Decommission
Closed, ResolvedPublic

Description

As per Jaime's and myself chat, db1019 can be decommissioned.
It is not pooled at this point.
I have silenced all the checks, downtimed the host and the notifications

Mentioned in SAL (#wikimedia-operations) [2016-09-21T09:26:19Z] <marostegui> Stopping mysql at db1019 for a few days as it will be decommissioned - T146265

This server was doing nothing but replicating, so mysql has been stopped:

MariaDB PRODUCTION s4 localhost (none) > show processlist;
+-----------+-----------------+------------------+--------------------+---------+---------+--------------------------------------------
| Id        | User            | Host             | db                 | Command | Time    | State
+-----------+-----------------+------------------+--------------------+---------+---------+--------------------------------------------
|         2 | event_scheduler | localhost        | NULL               | Daemon  |       3 | Waiting for next activation
| 657109251 | system user     |                  | NULL               | Connect |       0 | Slave has read all relay log; waiting for t
| 658776361 | system user     |                  | NULL               | Connect | 5874509 | Waiting for master to send event
| 661964706 | watchdog        | 10.64.0.15:49567 | mysql              | Sleep   |       4 |
| 661964708 | watchdog        | 10.64.0.15:49640 | information_schema | Sleep   |       2 |
| 662053624 | root            | localhost        | NULL               | Query   |       0 | init
+-----------+-----------------+------------------+--------------------+---------+---------+--------------------------------------------
6 rows in set (0.01 sec)

MariaDB PRODUCTION s4 localhost (none) > show slave hosts;
Empty set (0.00 sec)

Change 313990 had a related patch set uploaded (by Marostegui):
db-eqiad.php: db1019 is going to be decommissioned

https://gerrit.wikimedia.org/r/313990

Change 313990 merged by jenkins-bot:
db-eqiad.php: db1019 is going to be decommissioned

https://gerrit.wikimedia.org/r/313990

Mentioned in SAL (#wikimedia-operations) [2016-10-04T11:14:48Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Removing db1019 entry as it is going to be decommissioned - T146265 (duration: 00m 51s)

Change 313995 had a related patch set uploaded (by Marostegui):
db1019 is going to be decommisioned

https://gerrit.wikimedia.org/r/313995

Change 313995 merged by Marostegui:
db1019 is going to be decommisioned

https://gerrit.wikimedia.org/r/313995

I have created the HW request to get it removed: https://phabricator.wikimedia.org/T147309

Also @Volans and myself were unsure about whether it can be safely remove from the array host list in the .php files, so we left it there until we get confirmation that it can be removed from there too.

@jcrespo can you confirm if it can be deleted from there without breaking the site as the header states? :-)

Thanks

It still needs to be deleted from DNS now that I think about it.
We can probably do this at the same time it gets delete from the array of hosts of db-equiad|codfw.php files.

@jcrespo can you confirm if it can be deleted from there without breaking the site as the header states? :-)

Create a CR, add me as a reviewer- it will be easier, I can tell you there.

Change 314252 had a related patch set uploaded (by Marostegui):
db1019 is going to be decommissioned

https://gerrit.wikimedia.org/r/314252

Change 314252 merged by jenkins-bot:
mariadb: Decommission db1019

https://gerrit.wikimedia.org/r/314252

Mentioned in SAL (#wikimedia-operations) [2016-10-05T13:32:38Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: wmf-config/db-codfw.php Remove db1019 entries as it is going to be decommissioned - T146265 (duration: 00m 49s)

Change 315656 had a related patch set uploaded (by Marostegui):
Remove db1019

https://gerrit.wikimedia.org/r/315656

Change 315656 merged by jenkins-bot:
Remove db1019

https://gerrit.wikimedia.org/r/315656

This is ready to go.

Restricted Application added a subscriber: Southparkfan. · View Herald TranscriptNov 3 2016, 9:40 AM
jcrespo moved this task from Blocked external/Not db team to Done on the DBA board.Nov 3 2016, 9:40 AM

Change 334010 had a related patch set uploaded (by Dzahn):
site.pp, DHCP: remove db1019

https://gerrit.wikimedia.org/r/334010

Change 334014 had a related patch set uploaded (by Dzahn):
remove db1019, db1042 incl. mgmt

https://gerrit.wikimedia.org/r/334014

Change 334010 merged by Dzahn:
site.pp, DHCP: remove db1019, db1042

https://gerrit.wikimedia.org/r/334010

Mentioned in SAL (#wikimedia-operations) [2017-01-26T22:58:43Z] <mutante> db1019, db1042 - revoke puppet certs, delete salt keys, schedule icinga downtime, stop services (T149793, T146265)

Mentioned in SAL (#wikimedia-operations) [2017-01-27T19:52:40Z] <mutante> db1019 - shutdown -h now (T146265)

Mentioned in SAL (#wikimedia-operations) [2017-01-27T20:14:26Z] <mutante> db1029, db1042, analytics1015, analytics1026 - puppet node deactivate, remove from icinga, finish decom (T147313, T149793, T146265)

Change 334014 merged by Dzahn:
remove db1019, db1042, keep mgmt

https://gerrit.wikimedia.org/r/334014

Marostegui closed this task as Resolved.Feb 16 2017, 10:40 AM

I believe this is done