Create and package a Prometheus exporter based on the existing Diamond collector
Description
Description
Details
Details
Related Changes in Gerrit:
Customize query in gerrit
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | fgiunchedi | T177195 Reduce technical debt in metrics monitoring | |||
| Resolved | fgiunchedi | T177196 Port non-deprecated Diamond collectors to Prometheus | |||
| Resolved | MoritzMuehlenhoff | T182773 Create Prometheus exporter for wdqs-updater |
Event Timeline
Comment Actions
Change 398073 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Add Prometheus exporter to WDQS servers
Comment Actions
Change 398073 merged by Muehlenhoff:
[operations/puppet@production] Add Prometheus exporter to WDQS servers
Comment Actions
I updated the grafana dashboard as well: https://grafana-admin.wikimedia.org/dashboard/db/wikidata-query-service-prometheus?refresh=1m&orgId=1
Comment Actions
I'm reopening this. With the current exporter, the metrics go through the following chain:
jmx -> jolokia -> prometheus-wdqs-updater-exporter -> prometheus
Since all metrics we collect are available via JMX, we can do simpler:
jmx -> prometheus-jmx-exporter -> prometheus
This has the added advantage of publishing the standard JVM metrics in a standard way.
Comment Actions
Change 405887 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] wdqs: replace prometheus-wdqs-updater-exporter with prometheus-jmx-exporter
Comment Actions
Change 405888 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] wdqs: remove cleanup code after migrating to prometheus jmx exporter
Comment Actions
Change 405907 had a related patch set uploaded (by Gehel; owner: Gehel):
[wikidata/query/rdf@master] Disable jolokia agent.
Comment Actions
Jolokia agent is not loaded anymore with https://gerrit.wikimedia.org/r/405907. We could also remove all references to jolokia, remove it from the packaging. This would make sense as the monitoring tools should be a deployment concern, and not part of the application itself. That being said, we might have other users of wdqs-updater who rely on jolokia already. @Smalyshev what do you think?
Comment Actions
I think if somebody needs jolokia for their own reporting, they can add it back through the options. Jolokia is not part of the API, it's a monitoring thing, so I don't think we need to keep it around if we don't need it.
Comment Actions
Change 405887 merged by Gehel:
[operations/puppet@production] wdqs: replace prometheus-wdqs-updater-exporter with prometheus-jmx-exporter
Comment Actions
Change 407427 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] wdqs: fixed typo after introducing prometheus-jmx-exporter
Comment Actions
Change 407427 merged by Gehel:
[operations/puppet@production] wdqs: fixed typo after introducing prometheus-jmx-exporter
Comment Actions
Change 407431 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] wdqs: corrected source of jmx exporter configuration
Comment Actions
Change 407431 merged by Gehel:
[operations/puppet@production] wdqs: corrected source of jmx exporter configuration
Comment Actions
Change 405888 merged by Gehel:
[operations/puppet@production] wdqs: remove cleanup code after migrating to prometheus jmx exporter
Comment Actions
Change 405907 merged by jenkins-bot:
[wikidata/query/rdf@master] Disable jolokia agent.
Comment Actions
Migration to prometheus is completed, dashboards have been updated and diamond / graphite code has been removed.