Page MenuHomePhabricator

Create Prometheus exporter for wdqs-updater
Closed, ResolvedPublic

Description

Create and package a Prometheus exporter based on the existing Diamond collector

Event Timeline

MoritzMuehlenhoff created this task.

Change 398073 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Add Prometheus exporter to WDQS servers

https://gerrit.wikimedia.org/r/398073

Change 398073 merged by Muehlenhoff:
[operations/puppet@production] Add Prometheus exporter to WDQS servers

https://gerrit.wikimedia.org/r/398073

An exporter has been written, packaged and rolled out.

I'm reopening this. With the current exporter, the metrics go through the following chain:

jmx -> jolokia -> prometheus-wdqs-updater-exporter -> prometheus

Since all metrics we collect are available via JMX, we can do simpler:

jmx -> prometheus-jmx-exporter -> prometheus

This has the added advantage of publishing the standard JVM metrics in a standard way.

Change 405887 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] wdqs: replace prometheus-wdqs-updater-exporter with prometheus-jmx-exporter

https://gerrit.wikimedia.org/r/405887

Change 405888 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] wdqs: remove cleanup code after migrating to prometheus jmx exporter

https://gerrit.wikimedia.org/r/405888

Change 405907 had a related patch set uploaded (by Gehel; owner: Gehel):
[wikidata/query/rdf@master] Disable jolokia agent.

https://gerrit.wikimedia.org/r/405907

Jolokia agent is not loaded anymore with https://gerrit.wikimedia.org/r/405907. We could also remove all references to jolokia, remove it from the packaging. This would make sense as the monitoring tools should be a deployment concern, and not part of the application itself. That being said, we might have other users of wdqs-updater who rely on jolokia already. @Smalyshev what do you think?

I think if somebody needs jolokia for their own reporting, they can add it back through the options. Jolokia is not part of the API, it's a monitoring thing, so I don't think we need to keep it around if we don't need it.

Change 405887 merged by Gehel:
[operations/puppet@production] wdqs: replace prometheus-wdqs-updater-exporter with prometheus-jmx-exporter

https://gerrit.wikimedia.org/r/405887

Change 407427 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] wdqs: fixed typo after introducing prometheus-jmx-exporter

https://gerrit.wikimedia.org/r/407427

Change 407427 merged by Gehel:
[operations/puppet@production] wdqs: fixed typo after introducing prometheus-jmx-exporter

https://gerrit.wikimedia.org/r/407427

Change 407431 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] wdqs: corrected source of jmx exporter configuration

https://gerrit.wikimedia.org/r/407431

Change 407431 merged by Gehel:
[operations/puppet@production] wdqs: corrected source of jmx exporter configuration

https://gerrit.wikimedia.org/r/407431

Change 405888 merged by Gehel:
[operations/puppet@production] wdqs: remove cleanup code after migrating to prometheus jmx exporter

https://gerrit.wikimedia.org/r/405888

Change 405907 merged by jenkins-bot:
[wikidata/query/rdf@master] Disable jolokia agent.

https://gerrit.wikimedia.org/r/405907

Migration to prometheus is completed, dashboards have been updated and diamond / graphite code has been removed.