Programmatic generation of grafana dashboards
Closed, DeclinedPublic
Actions

Assigned To

None

Authored By

	fgiunchedi
	Jul 24 2017, 3:35 PM

Description

At the moment all of our grafana dashboards are hand-edited, with some committed to puppet as per T133392.
The manual editing approach is ok but tedious and error prone especially when editing multiple dashboards. I've come across https://github.com/weaveworks/grafanalib before and it seems it could be a useful starting point to be able to write grafana dashboards as code instead.

Other alternatives include a jsonnet-based one: https://github.com/grafana/grafonnet-lib

And yaml based: https://github.com/jakubplichta/grafana-dashboard-builder or https://docs.openstack.org/infra/grafyaml/index.html

Javascript based: https://github.com/uber/grafana-dash-gen

Details

	Project	Branch	Lines +/-	Subject
	operations/puppet	production	+267 -0	WIP grafana: host overview dashboard as code

Customize query in gerrit

Related Objects

Mentioned In: T178690: Better organization for SRE grafana dashboards
T175850: Spike: Enumerate remaining unported stats
Mentioned Here: T133392: save grafana dashboards in revision control / puppet

Event Timeline

fgiunchedi created this task.Jul 24 2017, 3:35 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 24 2017, 3:35 PM

OpenStack infrastructure has a python based utility to generate Grafana board based on a YAML DSL. That is similar to their Jenkins Job Builder used to generate jobs.

Basic usage: https://docs.openstack.org/infra/grafyaml/grafana-dashboard.html
Source code: https://git.openstack.org/cgit/openstack-infra/grafyaml

Their yaml files: https://git.openstack.org/cgit/openstack-infra/project-config/tree/grafana

The target definition seems to be one huge long line, but I am pretty sure they can be indented.

• ema added a subscriber: • ema.Jul 25 2017, 10:02 AM

fgiunchedi added a project: User-fgiunchedi.Aug 1 2017, 3:41 PM

fgiunchedi mentioned this in T175850: Spike: Enumerate remaining unported stats.Sep 15 2017, 8:36 AM

Krinkle added a project: Graphite.Nov 21 2017, 10:48 PM

Krinkle moved this task from Inbox to Grafana on the Graphite board.

BTW, +1 for this. It'd be especially cool if we applied the same puppet profile in labs and got the same grafana dashboards there.

hashar removed a subscriber: hashar.Jan 16 2018, 8:28 PM

fgiunchedi moved this task from Backlog to Doing on the User-fgiunchedi board.Feb 1 2018, 1:40 PM

fgiunchedi updated the task description. (Show Details)Feb 6 2018, 4:05 PM

fgiunchedi updated the task description. (Show Details)Mar 22 2018, 10:59 AM

fgiunchedi updated the task description. (Show Details)Apr 20 2018, 2:55 PM

fgiunchedi merged a task: T133392: save grafana dashboards in revision control / puppet.May 8 2018, 2:14 PM

• Imarlier added a subscriber: • Imarlier.May 15 2018, 2:18 PM

Change 442301 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] WIP grafana: host overview dashboard as code

https://gerrit.wikimedia.org/r/442301

gerritbot added a project: Patch-For-Review.Jul 4 2018, 4:45 PM

fgiunchedi mentioned this in T178690: Better organization for SRE grafana dashboards.Jul 4 2018, 4:46 PM

CDanis added a subscriber: CDanis.Nov 14 2018, 5:39 PM

fgiunchedi moved this task from Doing to Up next on the User-fgiunchedi board.Dec 13 2018, 3:41 PM

• Phabricator_maintenance moved this task from Backlog to Acknowledged on the SRE board.Jan 26 2019, 9:24 PM

fgiunchedi moved this task from Up next to Backlog on the User-fgiunchedi board.Oct 9 2019, 11:31 PM

fgiunchedi moved this task from Inbox to Backlog on the observability board.Dec 10 2019, 2:18 PM

Kormat added a subscriber: Kormat.May 4 2020, 1:04 PM

lmata added a subscriber: lmata.Feb 25 2021, 6:03 PM

lmata edited projects, added SRE Observability; removed observability.Jul 12 2021, 2:39 AM

Maintenance_bot added a project: observability.Jul 12 2021, 2:51 AM

lmata moved this task from Inbox to Backlog on the SRE Observability board.Jul 15 2021, 4:09 AM

lmata edited projects, added SRE Observability (FY2021/2022-Q1); removed observability, SRE Observability.Aug 9 2021, 1:07 AM

@herron this sounds like should be folded into the Grizzly work or closed at this point. If you have opinions on either of these options or new options let me know :-)

Sounds good, yes grizzly deploys the jsonnet/grafonnet approach outlined in the task description and good progress has been made putting that in place.

At this point I think it would be helpful to talk about next steps, guidelines and workflows for transitioning general dashboards into the jsonnet model, and how this should interact with puppet (re: T171482#3904056 and similar)

Change 442301 abandoned by Filippo Giunchedi:

[operations/puppet@production] WIP grafana: host overview dashboard as code

Reason:

Obsolete

https://gerrit.wikimedia.org/r/442301

Maintenance_bot removed a project: Patch-For-Review.Aug 23 2021, 9:10 AM

lmata edited projects, added Observability-Metrics; removed Graphite.Sep 15 2021, 2:32 PM

lmata moved this task from Inbox to Backlog on the Observability-Metrics board.

lmata removed a project: SRE Observability (FY2021/2022-Q1).Oct 6 2021, 4:03 AM

this is superseded by grizzly, which is already in production for SLO dashboarding

GRIZZLYYYYY?