This will obsolete the eventlogging-reporter.
Description
Description
Details
Details
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | • kevinator | T102225 {stag} EventLogging on Kafka | |||
| Resolved | Ottomata | T102831 Prep work for Eventlogging on Kafka {stag} | |||
| Resolved | Ottomata | T98781 Processor writes valid and invalid events to separate Kafka topics {stag} [13 points] | |||
| Resolved | akosiaris | T103381 Open tcp ports 8421 and 8422 to eventlog1001 to the Analytics VLAN | |||
| Resolved | Ottomata | T106254 Make EventLogging monitoring and alerts based on Kafka metrics {stag} [8 pts] | |||
| Resolved | Ottomata | T106581 Build 0.8.2.1 Kafka package and upgrade Kafka brokers | |||
| Declined | Ottomata | T98161 Build Kafka 0.8.1.1 package for Jessie and upgrade Brokers to Jessie. | |||
| Resolved | Ottomata | T103106 Create jmxtrans Jessie package | |||
| Resolved | Ottomata | T90640 Audit hyperthreading on analytics nodes. | |||
| Resolved | Ottomata | T111182 Fix auto_offset_reset bug in pykafka and build new .deb package {stag} [8 pts] |
Event Timeline
Comment Actions
Will this task solve https://phabricator.wikimedia.org/T106495?
If so, we should document them as dups.
Comment Actions
Hm, I'm not really sure. I think so, because these metrics would then be based on messages per second in a topic, which we get all at once from JMX. I could see a delay happening if the processor is backed up a lot, but, not a delay of metric collection.
Comment Actions
ToDo:
- Change eventlogging::monitoring::graphite so that is uses Kafka JMX metrics
- Undo eventlogging::monitoring::ganglia
- Undo role::eventlogging::reporter
- Undo role::eventlogging::consumer::graphite
Comment Actions
Change 236947 had a related patch set uploaded (by Ottomata):
Modify eventlogging graphite alerts so that they are based on kafka metrics
Comment Actions
Change 236947 merged by Ottomata:
Modify eventlogging graphite alerts so that they are based on kafka metrics
Comment Actions
Change 237188 had a related patch set uploaded (by Ottomata):
Remove eventlogging ZMQ based monitoring