Page MenuHomePhabricator

Convert prometheus-labs-targets to use nova API instead of wikitech's api.php
Closed, ResolvedPublic

Description

The ./modules/prometheus/files/usr/local/bin/prometheus-labs-targets script fetches the list of instances in a given project using wikitech.w.o/api.php, with region hardcoded to eqiad. This is obviously broken with neutron coming along and unsupported, we should switch to using nova API and its public readonly credentials instead.

More context from irc today with @aborrero

11:17  <arturo> godog: here is another simple example of using the nova API from python to fetch instances of a 
                project: ./modules/openstack/files/monitor/spreadcheck.py
11:17  <arturo>     client = novaclient.Client("1.1", project_id=project, **creds)
11:17  <arturo>     servers = client.servers.list()
11:17  <godog> nice, seems simple enough
11:18  <godog> and I should be able to hit nova api with readonly public credentials ?
11:18  <arturo> yes
11:19  <godog> awesome, thanks arturo for your help!
11:20  <arturo> that example script loads credentials from this file: ./modules/openstack/templates/monitor/spreadcheck-tools.yaml.erb

Event Timeline

fgiunchedi triaged this task as Normal priority.Jan 17 2019, 4:58 PM
fgiunchedi created this task.
fgiunchedi updated the task description. (Show Details)Jan 17 2019, 5:41 PM

Change 485193 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] prometheus: update prometheus-labs-targets to use keystone/nova clients

https://gerrit.wikimedia.org/r/485193

Change 485620 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] prometheus: move ::scripts to ::wmcs_scripts

https://gerrit.wikimedia.org/r/485620

Change 485621 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] prometheus: fetch WMCS targets from all regions

https://gerrit.wikimedia.org/r/485621

Change 485623 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] prometheus: use .format instead of % in prometheus-labs-targets

https://gerrit.wikimedia.org/r/485623

Change 485193 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: update prometheus-labs-targets to use keystone/nova clients

https://gerrit.wikimedia.org/r/485193

Change 485620 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: move ::scripts to ::wmcs_scripts

https://gerrit.wikimedia.org/r/485620

Change 485621 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: fetch WMCS targets from all regions

https://gerrit.wikimedia.org/r/485621

Change 485623 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: use .format instead of % in prometheus-labs-targets

https://gerrit.wikimedia.org/r/485623

Change 485635 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] prometheus: install wmcs_scripts dependencies from jessie-backports

https://gerrit.wikimedia.org/r/485635

Change 485635 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: install wmcs_scripts dependencies from jessie-backports

https://gerrit.wikimedia.org/r/485635

fgiunchedi closed this task as Resolved.Jan 21 2019, 2:07 PM
fgiunchedi claimed this task.

This is completed! Checked in tools targets are getting updated as expected:

root@tools-prometheus-02:~# ls -la /srv/prometheus/tools/targets/
total 32
drwxr-x--- 2 prometheus root       4096 Jan 21 14:07 .
drwxr-x--- 5 prometheus root       4096 Nov 20 10:34 ..
-rw-r--r-- 1 prometheus prometheus  105 Jan 21 14:00 etcd_flannel.yml
-rw-r--r-- 1 prometheus prometheus   93 Jan 21 14:00 etcd_k8s.yml
-rw-r--r-- 1 prometheus prometheus 7120 Jan 21 14:00 node_project.yml
-rw-r--r-- 1 prometheus prometheus 6652 Jan 21 14:00 ssh_banner.yml

Congratulations for managing your way out of the rabbit hole :-)