Page MenuHomePhabricator

Use profile and prometheus for role::kafka::main::broker
Closed, ResolvedPublic

Description

I'd like to reduce the number of moving parts for the upcoming main Kafka cluster upgrade. It will be easier to manage this upgrade if we switch as many things as we can before the actual Kafka version upgrade. This task is about applying profile::kafka::broker to Kafka main, while keeping the same version of Java and Kafka running there. This will enable prometheus based monitoring, and deprecate the jmxtrans/graphite monitoring.

Event Timeline

Ottomata created this task.Apr 23 2018, 6:48 PM
Ottomata triaged this task as Normal priority.

Change 428405 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Prep for profile::kafka::broker for main Kafka

https://gerrit.wikimedia.org/r/428405

Change 428405 merged by Ottomata:
[operations/puppet@production] Prep for profile::kafka::broker for main Kafka

https://gerrit.wikimedia.org/r/428405

Change 428410 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Use realm conditional in role::kafka::main::broker so we can test

https://gerrit.wikimedia.org/r/428410

Change 428410 merged by Ottomata:
[operations/puppet@production] Use realm conditional in role::kafka::main::broker so we can test

https://gerrit.wikimedia.org/r/428410

Change 428415 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Include proper profile::kafka::broker class in Cloud VPS

https://gerrit.wikimedia.org/r/428415

Change 428415 merged by Ottomata:
[operations/puppet@production] Include proper profile::kafka::broker class in Cloud VPS

https://gerrit.wikimedia.org/r/428415

Change 428419 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Only require java.security if declared (with Kafka ssl_enabled)

https://gerrit.wikimedia.org/r/428419

Change 428419 merged by Ottomata:
[operations/puppet@production] Only require java.security if declared (with Kafka ssl_enabled)

https://gerrit.wikimedia.org/r/428419

Change 428642 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] role::kafka::main - only include MirrorMaker in prod

https://gerrit.wikimedia.org/r/428642

Change 428642 merged by Ottomata:
[operations/puppet@production] role::kafka::main - only include MirrorMaker in prod

https://gerrit.wikimedia.org/r/428642

Change 428647 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Set default 'cluster' for profile::kafka::broker::monitoring

https://gerrit.wikimedia.org/r/428647

Change 428647 merged by Ottomata:
[operations/puppet@production] Set default 'cluster' for profile::kafka::broker::monitoring

https://gerrit.wikimedia.org/r/428647

Change 428648 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Set defaults for replica_maxlag broker monitoring thresholds

https://gerrit.wikimedia.org/r/428648

Change 428648 merged by Ottomata:
[operations/puppet@production] Set defaults for replica_maxlag broker monitoring thresholds

https://gerrit.wikimedia.org/r/428648

Change 428654 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Remove unneeded defaults from role/kafka/main.yaml hiera

https://gerrit.wikimedia.org/r/428654

Change 428654 merged by Ottomata:
[operations/puppet@production] Remove unneeded defaults from role/kafka/main.yaml hiera

https://gerrit.wikimedia.org/r/428654

Change 428656 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Use profile::kafka::broker for main-codfw

https://gerrit.wikimedia.org/r/428656

Change 428658 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Add kafka_cluster label to Prometheus Kafka metrics

https://gerrit.wikimedia.org/r/428658

Change 428658 merged by Ottomata:
[operations/puppet@production] Add kafka_cluster label to Prometheus Kafka metrics

https://gerrit.wikimedia.org/r/428658

Change 428681 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Use kafka_cluster label instead of cluster for broker alerts

https://gerrit.wikimedia.org/r/428681

Change 428681 merged by Ottomata:
[operations/puppet@production] Use kafka_cluster label instead of cluster for broker alerts

https://gerrit.wikimedia.org/r/428681

Change 428703 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Use different GC opts for Kafka in Java 7 vs Java 8

https://gerrit.wikimedia.org/r/428703

Change 428703 merged by Ottomata:
[operations/puppet@production] Use different GC opts for Kafka in Java 7 vs Java 8

https://gerrit.wikimedia.org/r/428703

Mentioned in SAL (#wikimedia-operations) [2018-04-24T17:42:16Z] <ottomata> temp disabling puppet on kafka200* to apply profile::kafka::broker in main-codfw T192831

Change 428656 merged by Ottomata:
[operations/puppet@production] Use profile::kafka::broker for main-codfw

https://gerrit.wikimedia.org/r/428656

Change 428711 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Use profile::kafka::broker in main-eqiad

https://gerrit.wikimedia.org/r/428711

Mentioned in SAL (#wikimedia-operations) [2018-04-24T18:54:11Z] <ottomata> temp disabling puppet and applying profile::kafka::broker on kafka100* T192831

Change 428711 merged by Ottomata:
[operations/puppet@production] Use profile::kafka::broker in main-eqiad

https://gerrit.wikimedia.org/r/428711

Change 428717 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Remove unused role::kafka::main::broker

https://gerrit.wikimedia.org/r/428717

Change 428826 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Fix kafka alert dashboard link

https://gerrit.wikimedia.org/r/428826

Change 428717 merged by Ottomata:
[operations/puppet@production] Remove unused role::kafka::main::broker

https://gerrit.wikimedia.org/r/428717

Change 428826 merged by Ottomata:
[operations/puppet@production] Fix kafka alert dashboard link

https://gerrit.wikimedia.org/r/428826

Ottomata moved this task from Next Up to Done on the Analytics-Kanban board.Apr 24 2018, 8:40 PM
fdans moved this task from Incoming to Kafka Work on the Analytics board.Apr 26 2018, 4:35 PM
Nuria closed this task as Resolved.May 8 2018, 10:45 PM
Vvjjkkii renamed this task from Use profile and prometheus for role::kafka::main::broker to weeaaaaaaa.Jul 1 2018, 1:14 AM
Vvjjkkii reopened this task as Open.
Vvjjkkii raised the priority of this task from Normal to High.
Vvjjkkii removed Ottomata as the assignee of this task.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed the point value for this task.
Vvjjkkii removed subscribers: Aklapper, gerritbot.
Samwilson renamed this task from weeaaaaaaa to Use profile and prometheus for role::kafka::main::broker .Jul 1 2018, 4:59 AM
Samwilson closed this task as Resolved.
Samwilson lowered the priority of this task from High to Normal.
Samwilson assigned this task to Ottomata.
Samwilson updated the task description. (Show Details)
Samwilson added subscribers: GerritBot, Aklapper.