As per title, all hosts running this role should be on Buster. Ideally by FQ2 FY20-21.
```
root@cumin1001:~# cumin 'P{O:netmon} and not P{F:lsbdistcodename = buster}'
2 hosts will be targeted:
netmon[1002,2001].wikimedia.org
DRY-RUN mode enabled, aborting
```
Current plan:
[x] Lower TTL for librenms/smokeping DNS records https://gerrit.wikimedia.org/r/c/operations/dns/+/615417
[x] Reimage netmon2001 with Buster, verify basic puppet runs are working as intended
[x] Flip netmon2001 to be the active server
[x] Sync RRDs in `/srv/librenms/rrd/`
[x] Sync rancid in `/var/lib/rancid/core`
[x] Set netmon2001 as `netmon_server` in puppet and netmon1002 as `netmon_server_failover` run puppet on both netmon1002 and netmon2001 https://gerrit.wikimedia.org/r/c/operations/puppet/+/616710
[x] Flip DNS for librenms and smokeping to netmon2001 https://gerrit.wikimedia.org/r/c/operations/dns/+/616709
[x] Verify librenms is working as intended: users can login, data is collected, alerts can be sent, etc
[] Verify rancid/smokeping are working as intended
[] Verify snmp-exporter is working as intended
[] Reimage netmon1002 with Buster
[] Flip back from netmon2001 to netmon1002
Issues identified:
1. Polling time for eqiad devices increased significantly due to the added latency. For the most populated rows (eqiad B and D) this means that occasionally poll times are >5 min, resulting in alerts and potentially missed data
1. snmp-exporter configuration will need adjusting due to version upgrade. I've temporarily downgraded snmp-exporter on netmon2001 to keep PDU data flowing into Prometheus
1. librenms web ui got significantly slower (from Europe at least) in part because of the added latency to reach codfw, in part because the database is still in eqiad