Page MenuHomePhabricator

Prometheus not collecting cloudelastic metrics
Closed, ResolvedPublic

Description

Perhaps i'm just not finding them, but it doesn't look like cloudelastic metrics are making it to prometheus. Our dashboards, even the generic ones like Cluster Overview and Host overview do not list the cloudelastic servers.

A quick check of things I could think of that have to be in place:

  • prometheus-node-exporter is running and listening on :9100
  • ferm looks to have appropriate rules allowing connection from prometheus machines:
    • &R_SERVICE(tcp, 9100, (@resolve((prometheus1003.eqiad.wmnet prometheus1004.eqiad.wmnet)) @resolve((prometheus1003.eqiad.wmnet prometheus1004.eqiad.wmnet), AAAA)));
  • fetching localhost:9100/metrics returns what looks to be appropriate metrics

Random guess at problems:

  • Could the *.wikimedia.org domain be throwing things off? Maybe eqiad prometheus only wants to talk to *.eqiad.wmnet?

Might require someone that can login to the prometheus servers and double check whats going on with it's configuration.

Event Timeline

Looked into this a little bit (on cloudelastic1001.wikimedia.org), no solution yet:

  • Verified with tcpdump over 10 minutes that nothing is calling port 9100 . Tcpdump of same port and time period on elastic1050 had requests from both prometheus servers on 1 minute intervals.
  • Prometheus generally decides what to poll via prometheus::class_config definitions looking up hosts via query to puppetdb.
  • Verified via /var/lib/puppet/client_data/catalog/cloudelastic1001.wikimedia.org.json that classes that are being looked for exist
  • Tried to verify via same file that $::site=='eqiad as expected and it looks likely, although it doesn't say definitively
  • puppetdb is unfortunately locked up, only accessible from cumin and puppetmasters so I can't test queries against it directly to figure out how cloudelastic differs

Change 528554 had a related patch set uploaded (by EBernhardson; owner: EBernhardson):
[operations/puppet@production] Define cloudelastic as a cluster in hieradata

https://gerrit.wikimedia.org/r/528554

Think i found it:

Cumin::Selector is defined with cluster="cloudelastic", site="eqiad"
Prometheus::Class_config (and friends) use the puppetlib get_clusters function, but cloudelastic is not configured in hiera as a valid cluster so never gets returned

Change 528554 merged by Gehel:
[operations/puppet@production] Define cloudelastic as a cluster in hieradata

https://gerrit.wikimedia.org/r/528554