Page MenuHomePhabricator

Set up a Prometheus server under the integration project
Open, NormalPublic

Description

The integration project will need its own Prometheus server for collection and storage of metrics related to the project instances and the CI cluster overall.

We can probably base the necessary ops/puppet changes off of the role::prometheus::beta module used for deployment-prometheus01—which currently serves as a collector and data source for Beta Cluster.

Event Timeline

dduvall triaged this task as Normal priority.Oct 1 2018, 9:12 PM
dduvall created this task.

See T182759: Add Prometheus exporter to Jenkins instances for the correct Prometheus target URLs of the Jenkins masters.

@fgiunchedi do you have any guidance to create a Prometheus instance on WMCS and have the exporters applied on all instances of a WMCS tenant?

@fgiunchedi do you have any guidance to create a Prometheus instance on WMCS and have the exporters applied on all instances of a WMCS tenant?

The first step is what @dduvall outlined, i.e. adapt what's being done for beta prometheus server. As for exporters, ./modules/prometheus/files/usr/local/bin/prometheus-labs-targets will periodically (re)configure Prometheus to poll node-exporter metrics from all instances of the project it is deployed onto. For additional exporters you can go a similar way than production, namely generate the necessary configuration from puppet exported resources (check out prometheus::class_config for an example)

HTH!