Page MenuHomePhabricator

decom netmon1001
Closed, ResolvedPublic

Description

netmon1001 was once on precise, was then upgraded to jessie (T125020) , then netmon1002 (T159756) and netmon2001 (T166180) have been setup to replace it and services have been migrated.

it can now be decom'ed. I assume final decom because HW warranty expiration was 2014-01-27

this is WMF3436, Dell R510 (https://racktables.wikimedia.org/index.php?page=object&tab=default&object_id=887)

netmon1001:

  • - all system services confirmed offline from production use
  • - set all icinga checks to maint mode/disabled while reclaim/decommmission takes place.
  • - remove system from all lvs/pybal active configuration
  • - any service group puppet/heira/dsh config removed
  • - remove site.pp (replace with role::spare if system isn't shut down immediately during this process.)

START NON-INTERRUPPTABLE STEPS

END NON-INTERRUPPTABLE STEPS

  • - system disks wiped (by onsite)
  • - system unracked and decommissioned (by onsite), update racktables with result
  • - switch port configration removed from switch once system is unracked.
  • - mgmt dns entries removed.

Event Timeline

Dzahn created this task.Jul 19 2017, 12:48 AM
Restricted Application added a project: Operations. · View Herald TranscriptJul 19 2017, 12:48 AM

Change 366178 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] netmon: remove librenms from netmon1001

https://gerrit.wikimedia.org/r/366178

Change 366178 merged by Dzahn:
[operations/puppet@production] netmon: remove librenms from netmon1001

https://gerrit.wikimedia.org/r/366178

Change 366180 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] prometheus: netmon1001 -> netmon1002 for snmp exporter

https://gerrit.wikimedia.org/r/366180

Change 366180 merged by Dzahn:
[operations/puppet@production] prometheus: netmon1001 -> netmon1002 for snmp exporter

https://gerrit.wikimedia.org/r/366180

Dzahn triaged this task as Normal priority.Jul 19 2017, 1:28 AM
Dzahn updated the task description. (Show Details)
Dzahn changed the task status from Open to Stalled.Jul 19 2017, 1:44 AM
Dzahn added a subscriber: ayounsi.

@ayounsi netmon1001 has been replaced by netmon1002. It seemed you were the only user who actually had data in a home dir there. I tar.gz'ed your home dir and copied to your new home on netmon1002 as home-ayounsi-netmon1001.tar.gz.

Dzahn changed the task status from Stalled to Open.Jul 19 2017, 1:44 AM
Dzahn changed the task status from Open to Stalled.Jul 19 2017, 4:51 PM

stalled for a moment please, librenms had to go back to it temp.

RobH assigned this task to Dzahn.Jul 19 2017, 4:54 PM
RobH added a subscriber: RobH.

Please feel free to assign to me once this is ready for the non-interrupt steps, thanks!

faidon raised the priority of this task from Normal to High.Jul 20 2017, 1:25 PM
faidon moved this task from Backlog to In progress on the observability board.

Change 366617 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] Revert "netmon1001: temp put librenms role back on it"

https://gerrit.wikimedia.org/r/366617

Dzahn changed the task status from Stalled to Open.Jul 20 2017, 7:25 PM

Change 366617 merged by Dzahn:
[operations/puppet@production] Revert "netmon1001: temp put librenms role back on it"

https://gerrit.wikimedia.org/r/366617

Change 366873 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] librenms: rsync direction netmon1002->netmon2001

https://gerrit.wikimedia.org/r/366873

Change 366873 merged by Dzahn:
[operations/puppet@production] librenms: rsync direction netmon1002->netmon2001

https://gerrit.wikimedia.org/r/366873

Dzahn reassigned this task from Dzahn to RobH.EditedJul 21 2017, 5:17 PM

This is ready now.

I did not remove from DHCP/netboot or DNS.

Change 366877 had a related patch set uploaded (by RobH; owner: RobH):
[operations/puppet@production] decom netmon1001

https://gerrit.wikimedia.org/r/366877

Change 366878 had a related patch set uploaded (by RobH; owner: RobH):
[operations/dns@master] decom of netmon1001 production dns

https://gerrit.wikimedia.org/r/366878

Change 366877 merged by RobH:
[operations/puppet@production] decom netmon1001

https://gerrit.wikimedia.org/r/366877

Change 366878 merged by RobH:
[operations/dns@master] decom of netmon1001 production dns

https://gerrit.wikimedia.org/r/366878

RobH reassigned this task from RobH to Cmjohnson.Jul 21 2017, 5:35 PM
RobH lowered the priority of this task from High to Normal.
RobH edited projects, added ops-eqiad; removed Patch-For-Review.
RobH updated the task description. (Show Details)
RobH moved this task from Backlog to Not urgent on the ops-eqiad board.
RobH moved this task from Not urgent to Decommission on the ops-eqiad board.

The disks have been wiped.

Cmjohnson closed this task as Resolved.Oct 12 2017, 5:06 PM
Cmjohnson updated the task description. (Show Details)