Page MenuHomePhabricator

Put netmon2002 in service
Closed, ResolvedPublic

Description

Host was racked in T313867 and needs to be put in production.

Prerequisites:

  • Apply the role::netmon to the netmon2002 host #854624.
  • Set netmon2002 as a passive server host #854625
  • Confirm the netmon2002 host has DB privileges
    • Open firewall port for the LibreNMS database #854951
    • Connect to the LibreNMS database
  • Determine what (if any) data should to be carried over from netmon2001 to netmon2002 (Also relevant for T322695.)
  • Sync RRDs in /srv/librenms/rrd/
  • Sync rancid in /var/lib/rancid/core and /var/lib/rancid/GIT
  • Add the netmon2002 host to the alertmanager API rw #854974
  • Add the netmon2002 instance as a ganeti rapi node #865707
  • Add the netmon2002 as a LibreNMS scap deploy target #865705

codfw failover:

  • Set netmon2002 the main instance in codfw #865711

Post deployment validation:

Event Timeline

Change 854625 had a related patch set uploaded (by Andrea Denisse; author: Andrea Denisse):

[operations/puppet@production] netmon: Put the netmon2002 as passive server

https://gerrit.wikimedia.org/r/854625

Change 854624 had a related patch set uploaded (by Andrea Denisse; author: Andrea Denisse):

[operations/puppet@production] netmon: Add regex to match all the netmon instances.

https://gerrit.wikimedia.org/r/854624

Change 854951 had a related patch set uploaded (by Andrea Denisse; author: Andrea Denisse):

[operations/puppet@production] netmon: Open LibreNMS port for netmon2002.

https://gerrit.wikimedia.org/r/854951

Change 854624 merged by Andrea Denisse:

[operations/puppet@production] netmon: Add regex to match all the netmon instances.

https://gerrit.wikimedia.org/r/854624

Change 854974 had a related patch set uploaded (by Andrea Denisse; author: Andrea Denisse):

[operations/puppet@production] netmon: Add netmon2002 to the alertmanager rw api

https://gerrit.wikimedia.org/r/854974

Change 856065 had a related patch set uploaded (by Andrea Denisse; author: Andrea Denisse):

[operations/dns@master] Lower the TTL for netbox for the migration.

https://gerrit.wikimedia.org/r/856065

Lower LibreNMS and Netbox TTL in preparation for the new instance

@fgiunchedi @andrea.denisse What has Netbox to do with the netmon work? Is that a typo? I've commented on the related CR too.

Change 856065 abandoned by Andrea Denisse:

[operations/dns@master] Lower the TTL for netbox for the migration.

Reason:

The patch is not required for the netmon2002 migration.

https://gerrit.wikimedia.org/r/856065

Change 854625 merged by Andrea Denisse:

[operations/puppet@production] netmon: Put the netmon2002 as passive server

https://gerrit.wikimedia.org/r/854625

Change 854974 merged by Andrea Denisse:

[operations/puppet@production] netmon: Add netmon2002 to the alertmanager rw api

https://gerrit.wikimedia.org/r/854974

Change 854951 merged by Andrea Denisse:

[operations/puppet@production] netmon: Open LibreNMS port for netmon2002.

https://gerrit.wikimedia.org/r/854951

Change 865705 had a related patch set uploaded (by Andrea Denisse; author: Andrea Denisse):

[operations/puppet@production] netmon: Add the netmon2002 as a LibreNMS scap deploy target

https://gerrit.wikimedia.org/r/865705

Change 865707 had a related patch set uploaded (by Andrea Denisse; author: Andrea Denisse):

[operations/puppet@production] netmon: Add the netmon2002 instance as a ganeti rapi node.

https://gerrit.wikimedia.org/r/865707

Change 865711 had a related patch set uploaded (by Andrea Denisse; author: Andrea Denisse):

[operations/puppet@production] netmon: Set netmon2002 the main instance in codfw

https://gerrit.wikimedia.org/r/865711

Change 865705 merged by Andrea Denisse:

[operations/puppet@production] netmon: Add the netmon2002 as a LibreNMS scap deploy target

https://gerrit.wikimedia.org/r/865705

Change 865707 merged by Andrea Denisse:

[operations/puppet@production] netmon: Add the netmon2002 instance as a ganeti rapi node.

https://gerrit.wikimedia.org/r/865707

Change 865711 merged by Andrea Denisse:

[operations/puppet@production] netmon: Set netmon2002 the main instance in codfw

https://gerrit.wikimedia.org/r/865711

andrea.denisse updated the task description. (Show Details)