Add RIPE atlas data to Prometheus
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	ayounsi
	Jun 12 2017, 3:49 PM

Description

Filippo made me aware of this great tool.

As we are hosting RIPE atlas anchors, the RIPE runs permanent measurements to our DCs.

They recently released a Prometheus exporter allowing us to import those measurements into Prometheus

See https://labs.ripe.net/Members/daniel_czerwonk/using-ripe-atlas-measurement-results-in-prometheus-with-atlas_exporter
and https://github.com/czerwonk/atlas_exporter for more informations.

Here are the measurements IDs we would need to query, to be configured under

static_configs:
  - targets:

CODFW

Type	v4	v6
Ping	1791210	1791212
Traceroute	1791209	1791211
HTTP	2841643	2841644

EQIAD

Type	v4	v6
Ping	1790945	1790947
Traceroute	1790944	1790946
HTTP	2841682	2841683

ULSFO

Type	v4	v6
Ping	1791307	1791309
Traceroute	1791306	1791308
HTTP	2841695	2841697

ESAMS and Singapore will need to be added when Anchor is setup.

Related Objects
Search...

Status	Assigned	Task
Resolved	CDanis	T167689 Add RIPE atlas data to Prometheus
Open	None	T251155 replace check_ripe_atlas Python script with a check_prometheus backed by atlasexporter data
Resolved	ayounsi	T251156 add traceroute measurements to RIPE Atlas prometheus data
Open	None	T283359 Create RIPE Atlas measurements against our authoritative DNS servers; alert on them

Event Timeline

ayounsi created this task.Jun 12 2017, 3:49 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 12 2017, 3:49 PM

@ayounsi awesome!

I'll outline here what needs to happen next:

Create a Debian package for atlas_exporter
Create the respective puppet module, similarly to existing exporters
Install the package and try the configuration on e.g. prometheus-beta, decide on things like how often to poll for data
Add the puppet role/profile/module to our Prometheus global instances (i.e. codfw/eqiad), I'd say running one atlas_exporter per prometheus machine is more than enough, each prometheus machine simply polls from localhost

@fgiunchedi: This task has no active projects associated. Should this be under User-fgiunchedi ? And/or SRE ? Or closed? Thanks in advance!

thanks @Aklapper ! fixed with operations and monitoring

MoritzMuehlenhoff triaged this task as Medium priority.Nov 8 2017, 3:42 PM

• Phabricator_maintenance moved this task from Backlog to Acknowledged on the SRE board.Jan 26 2019, 9:18 PM

faidon merged a task: T221964: RIPE Atlas data in Prometheus.Apr 26 2019, 2:33 PM

faidon added a subscriber: CDanis.

jbond subscribed.Apr 26 2019, 2:38 PM

colewhite subscribed.Jun 12 2019, 4:05 PM

fgiunchedi moved this task from Inbox to In progress on the observability board.Dec 10 2019, 2:18 PM

The steps outlined in Filippo's comment happened, with the difference that I chose to use the netmon* machines for this role.

The dashboard needs a bit more work, and we could look at also ingesting measurements other than ICMP pings, but this is done-ish.
https://grafana.wikimedia.org/d/K1qm1j-Wz/ripe-atlas

Thanks this is really nice!

It could be useful to add https://grafana.com/grafana/plugins/grafana-worldmap-panel and show the data on a map as well.

I'm interested in adding the traceroute hop count, for example to detect major routing changes. I looked a bit at the code, but would need to be walked through it to understand it fully.

Ladsgroup subscribed.Jan 7 2020, 8:52 AM

CDanis added a subtask: T251155: replace check_ripe_atlas Python script with a check_prometheus backed by atlasexporter data.Apr 27 2020, 3:13 PM

CDanis added a subtask: T251156: add traceroute measurements to RIPE Atlas prometheus data.