We have a new shiny Grafana deployment at https://grafana.wmcloud.org/ (thanks to @taavi!) and we want to migrate all dashboards from https://grafana-labs.wikimedia.org/ to the new server.
This is a epic/tracking task, please create sub-tasks for individual dashboards or groups of dashboards that can be migrated together.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | taavi | T307465 grafana-cloud: Browser access to Prometheus is deprecated | |||
| Resolved | fnegri | T333568 Move WMCS dashboards to grafana.wmcloud.org | |||
| Resolved | fnegri | T333567 Move ToolsDB Grafana dashboard to grafana.wmcloud.org |
There are a lot of dashboards in the old instance, though only a few ones are probably in active use. I'm not sure if there is a way to find out which ones are being used.
@taavi suggested in IRC we could take "a dump of all the dashboard configs", I'm not sure how to do it, but if it's easy maybe we could start from a full dump&restore of all the dashboards from the old to the new instance?
I imagine we might even search&replace the source IDs in the dump before restoring it, as I had to tweak those manually when I migrated the ToolsDB dashboard in T333567.
Are there better options that you can think of?
I used the following script to backup all the dashboards from https://grafana-labs.wikimedia.org/
for dash in $(curl -k -H "Authorization: Bearer $KEY" $HOST/api/search\?query\=\& |jq -r '.[] | .uid'); do curl -k -H "Authorization: Bearer $KEY" $HOST/api/dashboards/uid/$dash > dashboards/$dash.json done
I'm attaching the backup to this comment:
I wonder if instead of moving all of these to the new Grafana instance, we could move them upon request? There are probably some old dashboards that we don't really need to migrate.
Thanks! I was planning to do that myself, but you were faster it seems.
I wonder if instead of moving all of these to the new Grafana instance, we could move them upon request? There are probably some old dashboards that we don't really need to migrate.
+1, I'd like to use the new instance as an attempt to add some level of organization to the dashboards. Let's just upload that dump to https://download.wmcloud.org/?
+1, I'd like to use the new instance as an attempt to add some level of organization to the dashboards. Let's just upload that dump to https://download.wmcloud.org/?
Done! I'm not sure I like the idea of a generic collection of random undocumented files, but that'll do for now. :)
Do you think https://grafana-labs.wikimedia.org/ can be completely turned off at some point? Shall we send another email to cloud-announce with a planned shutdown date?
I was planning of doing something similar to what I did with graphite-labs.wikimedia.org, so have it redirect to the information page on Wikitech for a page and completely remove the domain a few months later once it's getting no traffic. The page should probably be updated with a link to the dump.
I already sent out one email for this a month ago announcing that we'll do it sometime this month. I can send a reminder when merging the patch for disabling the old Grafa instance.
As discussed above, we have decided not to migrate all the existing dashboards, but only to migrate a few important ones. If we ever need to resurrect one of the dashboards that were not migrated, we have a backup (linked above). All this is nicely recapped at https://wikitech.wikimedia.org/wiki/News/2023_Cloud_VPS_metrics_changes (thanks @taavi for writing that wiki page!)