Page MenuHomePhabricator

Make EventLogging monitoring and alerts based on Kafka metrics {stag} [8 pts]
Closed, ResolvedPublic

Description

This will obsolete the eventlogging-reporter.

Event Timeline

Ottomata created this task.Jul 19 2015, 5:54 AM
Ottomata claimed this task.
Ottomata raised the priority of this task from to Normal.
Ottomata updated the task description. (Show Details)
Ottomata added subscribers: gerritbot, Milimetric, Ottomata and 2 others.
Ottomata renamed this task from Make EventLogging alerts based on Kafka metrics to Make EventLogging alerts based on Kafka metrics {stag}.Jul 19 2015, 6:12 AM
Ottomata set Security to None.

Will this task solve https://phabricator.wikimedia.org/T106495?
If so, we should document them as dups.

Hm, I'm not really sure. I think so, because these metrics would then be based on messages per second in a topic, which we get all at once from JMX. I could see a delay happening if the processor is backed up a lot, but, not a delay of metric collection.

ggellerman raised the priority of this task from Normal to High.Jul 24 2015, 4:12 PM
ggellerman moved this task from Incoming to Low on the Analytics-Backlog board.
ggellerman moved this task from Low to Prioritized on the Analytics-Backlog board.
ggellerman lowered the priority of this task from High to Normal.Aug 7 2015, 4:10 PM
ggellerman moved this task from Prioritized to Medium on the Analytics-Backlog board.
kevinator renamed this task from Make EventLogging alerts based on Kafka metrics {stag} to Make EventLogging alerts based on Kafka metrics {stag} [8 pts].Aug 17 2015, 5:26 PM
kevinator moved this task from Prioritized to Tasked on the Analytics-Backlog board.
Ottomata renamed this task from Make EventLogging alerts based on Kafka metrics {stag} [8 pts] to Make EventLogging monitoring and alerts based on Kafka metrics {stag} [8 pts].Aug 25 2015, 2:51 PM
Ottomata added a comment.EditedAug 25 2015, 2:54 PM

ToDo:

  • Change eventlogging::monitoring::graphite so that is uses Kafka JMX metrics
  • Undo eventlogging::monitoring::ganglia
  • Undo role::eventlogging::reporter
  • Undo role::eventlogging::consumer::graphite

Change 236947 had a related patch set uploaded (by Ottomata):
Modify eventlogging graphite alerts so that they are based on kafka metrics

https://gerrit.wikimedia.org/r/236947

Ottomata moved this task from Next Up to In Progress on the Analytics-Kanban board.

Change 236947 merged by Ottomata:
Modify eventlogging graphite alerts so that they are based on kafka metrics

https://gerrit.wikimedia.org/r/236947

Change 237188 had a related patch set uploaded (by Ottomata):
Remove eventlogging ZMQ based monitoring

https://gerrit.wikimedia.org/r/237188

Change 237188 merged by Ottomata:
Remove eventlogging ZMQ based monitoring

https://gerrit.wikimedia.org/r/237188

Ottomata moved this task from In Code Review to Done on the Analytics-Kanban board.Sep 9 2015, 8:21 PM
kevinator closed this task as Resolved.Sep 10 2015, 7:54 PM