Page MenuHomePhabricator
Create Task
Maniphest T307465

grafana-cloud: Browser access to Prometheus is deprecated
Closed, ResolvedPublic
Actions

Description

I saw this when adding toolsbeta-prometheus to grafana-cloud:

image.png (221×837 px, 19 KB)

Grafana-cloud (aka grafana-labs) is hosted on production hardware (cloudmetrics100x.eqiad.wmnet). It has two main datasources:

  • graphite-labs, hosted on the same hardward and available publicly. This works now but graphite in general is being phased out.
  • Various Prometheus instances hosted on the cloud realm. Since the cloudmetrics hosts can't access production directly, these use the grafana setting to have browser javascript run the Prometheus queries (which is now deprecated upstream).

There's also an another Prometheus instance running on cloudmetrics hardware, but that one is only currently added to grafana.wm.o (not grafana-cloud).

Details

SubjectRepoBranchLines +/-
Disable access to grafana-labs/cloud.wm.ooperations/puppetproduction+4 -22
P:wmcs::metricsinfra::grafana: configure data sourcesoperations/puppetproduction+49 -0
P:wmcs::metricsinfra: add internal name for prometheusoperations/puppetproduction+1 -0
P:wmcs::metricsinfra: add haproxy config for grafanaoperations/puppetproduction+22 -0
P:metricsinfra: add profile and role for a Grafana serveroperations/puppetproduction+111 -0
P:grafana: stop provisioning home dashboardoperations/puppetproduction+1 -136
grafana: move dashboard config to a new defineoperations/puppetproduction+21 -8
P:grafana: make the logo file customizableoperations/puppetproduction+3 -1
P:grafana: move some profile declarations to rolesoperations/puppetproduction+3 -4
Customize query in gerrit

Related Objects
Search...

Event Timeline

• taavi created this task.May 3 2022, 2:14 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 3 2022, 2:14 PM
• taavi updated the task description. (Show Details)May 3 2022, 2:19 PM
• taavi added a project: SRE Observability.EditedMay 3 2022, 2:23 PM

Proposal:

Replace this Grafana instance with an another one running on Cloud VPS (in the metricsinfra project or in a separate one), as it is pretty much only used to query data for toolforge and cloud vps instances (and not the vps infrastructure in the production realm). Since graphite/statsd are already deprecated, they can remain on cloudmetrics at least for now. graphite-labs.wm.o is a public endpoint so dashboards will continue to work. Dashboards about the production infrastructure can be moved to the main Grafana instance.

see also: T207543: Move labmon (Graphite, StatsD) into a Cloud VPS

fgiunchedi subscribed.May 11 2022, 9:47 AM

+1 from me, thanks @Majavah for the proposal!

hashar mentioned this in T307655: Replacement needed for obsolete Diamond/Graphite monitoring of integration instances.Jun 16 2022, 3:08 PM
lmata moved this task from Inbox to Radar on the SRE Observability board.Jun 24 2022, 5:14 PM
gerritbot added a comment.Dec 19 2022, 1:00 PM

Change 869208 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] P:grafana: move some profile declarations to roles

https://gerrit.wikimedia.org/r/869208

gerritbot added a project: Patch-For-Review.Dec 19 2022, 1:00 PM

Change 869209 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] P:grafana: make the logo file customizable

https://gerrit.wikimedia.org/r/869209

gerritbot added a comment.Dec 19 2022, 1:00 PM

Change 869210 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] P:metricsinfra: add profile and role for a Grafana server

https://gerrit.wikimedia.org/r/869210

gerritbot added a comment.Dec 19 2022, 1:00 PM

Change 869211 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] P:wmcs::metricsinfra: add haproxy config for grafana

https://gerrit.wikimedia.org/r/869211

gerritbot added a comment.Dec 25 2022, 10:16 AM

Change 871289 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] grafana: move dashboard config to a new define

https://gerrit.wikimedia.org/r/871289

gerritbot added a comment.Dec 25 2022, 10:16 AM

Change 871290 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] P:grafana: stop provisioning home dashboard

https://gerrit.wikimedia.org/r/871290

gerritbot added a comment.Dec 25 2022, 10:16 AM

Change 871291 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] P:wmcs::metricsinfra: add internal name for prometheus

https://gerrit.wikimedia.org/r/871291

gerritbot added a comment.Dec 25 2022, 10:16 AM

Change 871292 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] P:wmcs::metricsinfra::grafana: configure data sources

https://gerrit.wikimedia.org/r/871292

gerritbot added a comment.Jan 3 2023, 11:30 AM

Change 869208 merged by Filippo Giunchedi:

[operations/puppet@production] P:grafana: move some profile declarations to roles

https://gerrit.wikimedia.org/r/869208

gerritbot added a comment.Jan 3 2023, 11:37 AM

Change 869209 merged by Filippo Giunchedi:

[operations/puppet@production] P:grafana: make the logo file customizable

https://gerrit.wikimedia.org/r/869209

gerritbot added a comment.Jan 3 2023, 4:54 PM

Change 871290 merged by Herron:

[operations/puppet@production] P:grafana: stop provisioning home dashboard

https://gerrit.wikimedia.org/r/871290

TheresNoTime subscribed.Jan 4 2023, 10:21 PM
fnegri subscribed.Jan 5 2023, 3:11 PM
fnegri edited projects, added cloud-services-team; removed cloud-services-team (Kanban).Jan 18 2023, 6:39 PM
fnegri moved this task from Kanban to Inbox on the cloud-services-team board.
gerritbot added a comment.Feb 1 2023, 9:21 AM

Change 869210 merged by David Caro:

[operations/puppet@production] P:metricsinfra: add profile and role for a Grafana server

https://gerrit.wikimedia.org/r/869210

gerritbot added a comment.Feb 1 2023, 9:21 AM

Change 869211 merged by David Caro:

[operations/puppet@production] P:wmcs::metricsinfra: add haproxy config for grafana

https://gerrit.wikimedia.org/r/869211

gerritbot added a comment.Feb 1 2023, 9:21 AM

Change 871291 merged by David Caro:

[operations/puppet@production] P:wmcs::metricsinfra: add internal name for prometheus

https://gerrit.wikimedia.org/r/871291

gerritbot added a comment.Feb 1 2023, 9:21 AM

Change 871292 merged by David Caro:

[operations/puppet@production] P:wmcs::metricsinfra::grafana: configure data sources

https://gerrit.wikimedia.org/r/871292

fnegri changed the task status from Open to In Progress.Feb 1 2023, 10:14 AM
fnegri triaged this task as Medium priority.
fnegri edited projects, added cloud-services-team (FY2022/2023-Q3); removed cloud-services-team.
fnegri moved this task from Backlog to In progress on the cloud-services-team (FY2022/2023-Q3) board.
• taavi claimed this task.Feb 1 2023, 11:26 AM
• taavi added a comment.Feb 5 2023, 4:49 PM

Status update: https://grafana.wmcloud.org is now running. Next step is to move over the dashboards.

• taavi mentioned this in T317887: Upgrade to Grafana 9.Feb 8 2023, 7:01 PM
fnegri added a comment.Mar 30 2023, 2:43 PM

I created T333568 to track the dashboard migration work, hoping it can be prioritized for next quarter (Q4).

fnegri added a subtask: T333568: Move WMCS dashboards to grafana.wmcloud.org.Mar 30 2023, 2:43 PM
fnegri moved this task from FY2022/2023-Q3 to Soon! on the cloud-services-team board.Apr 12 2023, 5:06 PM
fnegri edited projects, added cloud-services-team; removed cloud-services-team (FY2022/2023-Q3).
bd808 moved this task from Backlog to In Progress on the Toolforge board.Apr 27 2023, 11:24 PM
• taavi mentioned this in T336774: Current status of cloudmetrics and its components.May 16 2023, 3:27 PM
gerritbot added a comment.Aug 10 2023, 7:49 PM

Change 947933 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] Disable access to grafana-labs/cloud.wm.o

https://gerrit.wikimedia.org/r/947933

gerritbot added a comment.Aug 14 2023, 10:04 AM

Change 947933 merged by Filippo Giunchedi:

[operations/puppet@production] Disable access to grafana-labs/cloud.wm.o

https://gerrit.wikimedia.org/r/947933

fnegri closed subtask T333568: Move WMCS dashboards to grafana.wmcloud.org as Declined.Sep 22 2023, 5:35 PM
• taavi closed this task as Resolved.Sep 22 2023, 5:36 PM
fnegri changed the status of subtask T333568: Move WMCS dashboards to grafana.wmcloud.org from Declined to Resolved.Oct 3 2023, 6:00 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits