Page MenuHomePhabricator

Update CiviCRM ganglia bits to point to Prometheus
Closed, ResolvedPublic2 Estimated Story Points

Description

Ganglia is going away, and we have a couple of things that talk directly to it.

Here is the replacement:
https://wikitech.wikimedia.org/w/index.php?title=Prometheus

Event Timeline

DStrine moved this task from Triage to Q3 2021-2022 on the Fundraising-Backlog board.

PHP client library: https://github.com/Jimdo/prometheus_client_php
Metric types: https://prometheus.io/docs/concepts/metric_types/

All we're using it for is to count donations per gateway and overall at the end of each donation queue consumer run. Sounds like the 'counter' metric is just fine.
The client library says it uses redis for client-side aggregation. Any reason we can't use the same instance as we do for queues?

Change 379442 had a related patch set uploaded (by Ejegg; owner: Ejegg):
[wikimedia/fundraising/crm@master] WIP: Prometheus metrics reporting

https://gerrit.wikimedia.org/r/379442

Change 379806 had a related patch set uploaded (by Ejegg; owner: Ejegg):
[wikimedia/fundraising/crm@master] Delete ganglia_reporter, use new module

https://gerrit.wikimedia.org/r/379806

Ejegg set the point value for this task to 2.
Ejegg moved this task from Backlog to Review on the Fundraising Sprint RadioActivewear board.

Change 379442 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Prometheus metrics reporting

https://gerrit.wikimedia.org/r/379442

Change 379806 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Delete ganglia_reporter, use new module

https://gerrit.wikimedia.org/r/379806

@Jgreen or @cwdent : Where should we be writing the Prometheus files?

@Jgreen or @cwdent : Where should we be writing the Prometheus files?

Please write to /var/spool/prometheus/*.prom and just make sure they're readable by user prometheus. The dir has a group sticky bit, hopefully that will help.

Change 380799 had a related patch set uploaded (by Ejegg; owner: Ejegg):
[wikimedia/fundraising/crm@master] Prometheus: add extension, update default dir

https://gerrit.wikimedia.org/r/380799

Change 380799 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Prometheus: add extension, update default dir

https://gerrit.wikimedia.org/r/380799

OK, the new file queue2civicrm.prom is appearing in that directory. Do we have to do anything more to make the stats show up in the dashboards?