Change Details

We have Ganeti clusters availabile in PoPs now, thus we should move Prometheus to a Ganeti VM and off the bastions. Outline of steps per site: [x] Provision Prometheus VM (8GB ram / 128GB disk to start with) [] Add VM hostname to `prometheus_nodes` and `prometheus_all_nodes` [] Stop prometheus on bastion and VM + rsync `/srv/prometheus` bastion -> VM + restart Prometheus on VM and bastion [] Flip DNS name `prometheus.svc` to point to the VM [] Reimage bastion

We have Ganeti clusters availabile in PoPs now, thus we should move Prometheus to a Ganeti VM and off the bastions. Outline of steps per site: [x] Provision Prometheus VM (8GB ram / 128GB disk to start with) [x] Add VM hostname to `prometheus_nodes` and `prometheus_all_nodes` [] Stop prometheus on bastion and VM + rsync `/srv/prometheus` bastion -> VM + restart Prometheus on VM and bastion [] Flip DNS name `prometheus.svc` to point to the VM [] Reimage bastion

We have Ganeti clusters availabile in PoPs now, thus we should move Prometheus to a Ganeti VM and off the bastions. Outline of steps per site: [x] Provision Prometheus VM (8GB ram / 128GB disk to start with) [x] Add VM hostname to `prometheus_nodes` and `prometheus_all_nodes` [] Stop prometheus on bastion and VM + rsync `/srv/prometheus` bastion -> VM + restart Prometheus on VM and bastion [] Flip DNS name `prometheus.svc` to point to the VM [] Reimage bastion