This will ease the Kafka cluster upgrade(s), as we can run a 0.9 MirrorMaker using --new.consumer to consume from a 1.x cluster and produce to a 0.9 cluster. We can then upgrade Kafka main and get a stable 1.x MirrorMaker for main -> jumbo before proceeding with the analytics to jumbo Kafka client migrations.
This will also allow us to remove old graphite based MirrorMaker dashboards and metrics.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Ottomata | T167039 Upgrade Kafka on main cluster with security features | |||
| Resolved | Ottomata | T192387 Switch to --new.consumer for main -> analytics MirrorMaker |
Change 427400 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Split role::kafka::analytics into different roles so we can apply different hiera
Change 427400 merged by Ottomata:
[operations/puppet@production] Split role::kafka::analytics into 2 roles so we can apply different hiera
Change 427406 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Use --new.consumer for main-eqiad -> analytics MirrorMaker
Mentioned in SAL (#wikimedia-operations) [2018-04-23T14:10:19Z] <ottomata> switching main -> analytics MirrorMaker to --new.consumer (temporarily stopping puppet on kafka101[234]) https://phabricator.wikimedia.org/T192387
Mentioned in SAL (#wikimedia-analytics) [2018-04-23T14:10:24Z] <ottomata> switching main -> analytics MirrorMaker to --new.consumer (temporarily stopping puppet on kafka101[234]) https://phabricator.wikimedia.org/T192387
Change 427406 merged by Ottomata:
[operations/puppet@production] Use --new.consumer for main-eqiad -> analytics MirrorMaker
Change 428350 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Add prometheus based alerts for main-eqiad -> analytics eqiad
Change 428350 merged by Ottomata:
[operations/puppet@production] Add prometheus based alerts for main-eqiad -> analytics eqiad