Page MenuHomePhabricator

Provide authenticated access to Prometheus native web interface
Open, Needs TriagePublic

Description

Prometheus instances are proxied by apache on the machine it is running on. At the moment the way to access the native interface (e.g. for easier data/query exploration) is through an ssh tunnel (e.g. ssh prometheus1003.eqiad.wmnet -L8000:localhost:80 and then http://localhost:8000/ops/graph) but it'd be more convenient to expose the web interface behind web-misc and HTTP auth instead.

  • Setup apache LDAP auth for SRE (or NDA) access
  • Setup external DNS names
  • Setup misc-web HTTP routing

Details

Related Gerrit Patches:
operations/puppet : productionprometheus::web to apache

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 17 2016, 11:36 PM

re: nginx+ldap, it looks like the ldap auth module isn't included, though we can use pam auth for nginx and libpam-ldap as ldap client

faidon added a subscriber: faidon.Dec 9 2016, 4:46 PM

I don't think we should mess with the system's PAM config for this -- that's going to be a dangerous change, especially in the long run.

Change 377332 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] prometheus::web to apache

https://gerrit.wikimedia.org/r/377332

Change 377332 merged by Andrew Bogott:
[operations/puppet@production] prometheus::web to apache

https://gerrit.wikimedia.org/r/377332

fgiunchedi updated the task description. (Show Details)Feb 22 2018, 10:24 AM
fgiunchedi added a project: observability.
fgiunchedi moved this task from Backlog to Up next on the User-fgiunchedi board.May 11 2018, 10:10 AM
fgiunchedi moved this task from Up next to Backlog on the User-fgiunchedi board.Oct 9 2019, 11:31 PM
jbond added a subscriber: jbond.Mon, Dec 2, 11:55 AM
jbond added a comment.Mon, Dec 2, 11:57 AM

Im tempted to add this directly to apereo cas (time permitting) however im curious what you had in mind for the service domain names considering we need one for each codfw and eqiad?

Something like:

https://prometheous.codfw.wikimedia.org/
https://prometheous.eqiad.wikimedia.org/

or did you have something else in mind?

Im tempted to add this directly to apereo cas (time permitting) however im curious what you had in mind for the service domain names considering we need one for each codfw and eqiad?
Something like:

https://prometheous.codfw.wikimedia.org/
https://prometheous.eqiad.wikimedia.org/

or did you have something else in mind?

I think something like that would work great!

A braindump / couple of notes:

  • We'll need records/entries for all sites not only codfw/eqiad
  • Reverse-proxying from the public hostname to prometheus.svc.$site.wmnet should be enough but https will be needed on the apache side
  • We'll likely need a redirect say from / to /ops (sort of the default anyways, exists in all sites) or a landing page to list all options/instances available
fgiunchedi moved this task from Inbox to Backlog on the observability board.Tue, Dec 10, 2:19 PM