Page MenuHomePhabricator

Port redis statistics to Prometheus
Closed, ResolvedPublic

Description

Import a redis exporter (like https://github.com/oliver006/redis_exporter)

Event Timeline

I've tried running the above exporter in labs and mc2001, results for the latter are in P4446. Note the addr label is superfluous and can be discarded when prometheus polls (https://github.com/oliver006/redis_exporter/issues/32)

Change 325466 had a related patch set uploaded (by Filippo Giunchedi):
prometheus: add redis_exporter class

https://gerrit.wikimedia.org/r/325466

fgiunchedi closed this task as Resolved.Oct 5 2017, 3:56 PM
fgiunchedi claimed this task.

Will do as part of T177196

fgiunchedi reopened this task as Open.Oct 19 2017, 3:36 PM
fgiunchedi renamed this task from Port redis statistics from ganglia to prometheus to Port redis statistics to Prometheus.

Mentioned in SAL (#wikimedia-releng) [2017-10-27T13:11:30Z] <godog> provision deployment-redis{03,04} with stretch - T148637

Change 386869 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] hieradata: add redis stretch deployment-prep instances

https://gerrit.wikimedia.org/r/386869

Change 386870 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] redis: add stretch support

https://gerrit.wikimedia.org/r/386870

fgiunchedi moved this task from Backlog to Doing on the User-fgiunchedi board.

Change 386870 merged by Filippo Giunchedi:
[operations/puppet@production] redis: add stretch support

https://gerrit.wikimedia.org/r/386870

Change 325466 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: add redis_exporter class and profile

https://gerrit.wikimedia.org/r/325466

Change 390255 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] prometheus: fix notify prometheus-redis-exporter

https://gerrit.wikimedia.org/r/390255

Change 390255 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: fix notify prometheus-redis-exporter

https://gerrit.wikimedia.org/r/390255

Change 390260 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] prometheus: have multi-instance redis-exporter running, stop default one

https://gerrit.wikimedia.org/r/390260

Change 390260 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: have multi-instance redis-exporter running, stop default one

https://gerrit.wikimedia.org/r/390260

Change 390262 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] prometheus: switch to systemd::service

https://gerrit.wikimedia.org/r/390262

Change 390262 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: switch to systemd::service

https://gerrit.wikimedia.org/r/390262

Change 390263 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] prometheus: pass -redis.addr to redis-exporter

https://gerrit.wikimedia.org/r/390263

Change 390263 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: pass -redis.addr to redis-exporter

https://gerrit.wikimedia.org/r/390263

Change 391024 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] prometheus: add redis jobs

https://gerrit.wikimedia.org/r/391024

Change 391024 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: add redis jobs

https://gerrit.wikimedia.org/r/391024

Change 391834 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] prometheus: drop addr/alias redis_exporter labels

https://gerrit.wikimedia.org/r/391834

Mentioned in SAL (#wikimedia-operations) [2017-11-16T15:10:39Z] <godog> upgrade prometheus-redis-exporter to 0.13-1 - T148637

Change 391834 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: drop addr/alias redis_exporter labels

https://gerrit.wikimedia.org/r/391834

Change 392407 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] prometheus: fix relabeling for redis jobs

https://gerrit.wikimedia.org/r/392407

Change 392407 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: fix relabeling for redis jobs

https://gerrit.wikimedia.org/r/392407

Change 393605 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] redis: use hostname not fqdn in redis_exporter

https://gerrit.wikimedia.org/r/393605

Change 393606 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] profile: add redis_exporter to redis multidc

https://gerrit.wikimedia.org/r/393606

Change 393605 merged by Filippo Giunchedi:
[operations/puppet@production] redis: use hostname not fqdn in redis_exporter

https://gerrit.wikimedia.org/r/393605

Change 393606 merged by Filippo Giunchedi:
[operations/puppet@production] profile: add redis_exporter to redis multidc

https://gerrit.wikimedia.org/r/393606

Change 393794 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] role: split prometheus redis jobs

https://gerrit.wikimedia.org/r/393794

Change 393794 merged by Filippo Giunchedi:
[operations/puppet@production] role: split prometheus redis jobs

https://gerrit.wikimedia.org/r/393794

All redis (except for ores') have their metrics being pulled now!

Change 395563 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] profile: add redis_exporter to ores::redis

https://gerrit.wikimedia.org/r/395563

Change 395563 merged by Filippo Giunchedi:
[operations/puppet@production] profile: add redis_exporter to ores::redis

https://gerrit.wikimedia.org/r/395563

Change 395569 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] prometheus: add ores redis job

https://gerrit.wikimedia.org/r/395569

Change 395569 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: add ores redis job

https://gerrit.wikimedia.org/r/395569

Change 398871 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] prometheus: recording rules for redis

https://gerrit.wikimedia.org/r/398871

Change 398871 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: recording rules for redis

https://gerrit.wikimedia.org/r/398871

I "promoted" (renamed) the prometheus dashboard to "redis" and the previous to "redis-graphite": https://grafana.wikimedia.org/dashboard/db/redis cc @elukey @MoritzMuehlenhoff @Joe for feedback

fgiunchedi closed this task as Resolved.Dec 21 2017, 9:44 AM

All done! The dashboards will likely need some tuning but metrics are there.

Krenair added a subscriber: Krenair.Jun 5 2018, 7:45 PM

It has come up while reviewing cherry-picks used in deployment-prep that https://gerrit.wikimedia.org/r/386869 was not merged