Page MenuHomePhabricator

Upgrade grafana to 6.x
Open, LowPublic

Description

Grafana 6 adds a bunch of cool new stuff:

  • "Explore", a UI for playing around with monitoring data, and for exploring what metrics are in Prometheus. It looks ideal for either constructing queries for a new console, or for doing incident response across multiple systems (and then linking in your postmortem). https://grafana.com/docs/features/explore/
  • "ad hoc filtering for Prometheus", which looks like a big improvement on the template variable stuff we do on many consoles right now. https://grafana.com/docs/guides/whats-new-in-v6-1/
  • better panel editor UI

Things to watch out for per https://grafana.com/docs/installation/upgrading/#upgrading-to-v6-0

Plan is similar to last time:

  • create new Ganeti VM grafana1002.eqiad.wmnet -- will attempt using the new automation for such, and probably also try using buster
  • point grafana-beta.wikimedia.org to that host
  • copy a snapshot of the database and ask the same groups as in T210416 to test
  • pick a time to make -beta the new normal (should be just a few minutes of readonly)

Fundraising firewall should not be a concern this time as they have their own grafana now.

It's also worth looking at testing the upgrade on grafana-labs if they're interested

Event Timeline

CDanis created this task.Apr 12 2019, 4:55 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 12 2019, 4:55 PM
CDanis updated the task description. (Show Details)Apr 15 2019, 3:15 PM
CDanis moved this task from Backlog to Up next on the observability board.Apr 29 2019, 3:16 PM
Peter added a subscriber: Peter.Jun 14 2019, 5:05 AM
Peter added a comment.Jun 14 2019, 5:10 AM

If we could upgrade to 6.2.x that would be great. I've been using it for my projects for a while and the lazy loading of panels out of view really makes large dashboards working better (and faster). We have a couple of dashboards in the performance team that are really slow.

CDanis renamed this task from Upgrade grafana to 6.1 to Upgrade grafana to 6.x.Jun 24 2019, 3:12 PM
bd808 added a subscriber: bd808.

It's also worth looking at testing the upgrade on grafana-labs if they're interested

That would be great! And it would help address T226108: https://grafana-labs.wikimedia.org is a different version than https://grafana.wikimedia.org

CDanis triaged this task as Low priority.Jul 22 2019, 3:15 PM

so far work on dbctl has taken priority but I should have time for this in another few weeks.

Change 539894 had a related patch set uploaded (by CDanis; owner: CDanis):
[operations/dns@master] grafana1002: new VM for grafana 6.x

https://gerrit.wikimedia.org/r/539894

Change 539894 merged by CDanis:
[operations/dns@master] grafana1002: new VM for grafana 6.x

https://gerrit.wikimedia.org/r/539894

Change 539913 had a related patch set uploaded (by CDanis; owner: CDanis):
[operations/puppet@production] grafana1002: autoinstall/netboot/role data

https://gerrit.wikimedia.org/r/539913

Change 539913 merged by CDanis:
[operations/puppet@production] grafana1002: autoinstall/netboot/role data

https://gerrit.wikimedia.org/r/539913

Change 543861 had a related patch set uploaded (by CDanis; owner: CDanis):
[operations/puppet@production] grafana1002: answer for grafana-beta.wikimedia.org

https://gerrit.wikimedia.org/r/543861

Change 543861 merged by CDanis:
[operations/puppet@production] grafana1002: answer for grafana-beta.wikimedia.org

https://gerrit.wikimedia.org/r/543861

Change 543862 had a related patch set uploaded (by CDanis; owner: CDanis):
[operations/puppet@production] grafana-beta.wikimedia.org: point to grafana1002

https://gerrit.wikimedia.org/r/543862

Change 543862 merged by CDanis:
[operations/puppet@production] grafana-beta.wikimedia.org: point to grafana1002

https://gerrit.wikimedia.org/r/543862

Change 544023 had a related patch set uploaded (by CDanis; owner: CDanis):
[operations/puppet@production] trafficserver: actually map grafana-beta to grafana1002

https://gerrit.wikimedia.org/r/544023

Change 544023 merged by CDanis:
[operations/puppet@production] trafficserver: actually map grafana-beta to grafana1002

https://gerrit.wikimedia.org/r/544023

Change 544029 had a related patch set uploaded (by CDanis; owner: CDanis):
[operations/puppet@production] trafficserver: add never-cache for grafana1002

https://gerrit.wikimedia.org/r/544029

Change 544029 merged by CDanis:
[operations/puppet@production] trafficserver: add never-cache for grafana1002

https://gerrit.wikimedia.org/r/544029

CDanis updated the task description. (Show Details)Mon, Oct 21, 2:01 PM