Change Details

During the parent task we ran into an issue where Grafana failovers also require moving hiera variables around, which isn't expected and we've falled twice already into this pitfall. The procedure is documented now on wikitech thanks to @andrea.denisse, which is good and we should be working on making switchovers even simpler. This is the list of actions I believe we should take * [ ] Move grafana to discovery records (T356386) * [ ] Make sure we serve all grafana vhosts from all hosts, instead of the grafana / grafana-next distinction we have now * [ ] Protect the standby host(s) by refusing write actions to grafana API at the apache level

During the parent task we ran into an issue where Grafana failovers also require moving hiera variables around, which isn't expected and we've falled twice already into this pitfall. The procedure is documented now on wikitech thanks to @andrea.denisse, which is good and we should be working on making switchovers even simpler. This is the list of actions I believe we should take * [ ] Move grafana to discovery records (T356386) * [ ] Make sure we serve all grafana vhosts from all hosts, instead of the grafana / grafana-next distinction we have now * [ ] Protect the standby host(s) by refusing write actions to grafana API at the apache level * [ ] Make sure the sync timers follow the active/passive host, bonus points if this happens without a patch to puppet

During the parent task we ran into an issue where Grafana failovers also require moving hiera variables around, which isn't expected and we've falled twice already into this pitfall. The procedure is documented now on wikitech thanks to @andrea.denisse, which is good and we should be working on making switchovers even simpler. This is the list of actions I believe we should take * [ ] Move grafana to discovery records (T356386) * [ ] Make sure we serve all grafana vhosts from all hosts, instead of the grafana / grafana-next distinction we have now * [ ] Protect the standby host(s) by refusing write actions to grafana API at the apache level * [ ] Make sure the sync timers follow the active/passive host, bonus points if this happens without a patch to puppet