Page MenuHomePhabricator

Enable TLS and authorization for cross DC MirrorMaker
Closed, ResolvedPublic8 Estimated Story Points

Description

Now that T193778: SSL and inter broker encryption for Kafka main is done, we can enable TLS for MirrorMaker. Once that is done, we can re-enable cross DC mirroring for job queue and change prop topics.

Event Timeline

Ottomata created this task.May 31 2018, 3:36 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 31 2018, 3:36 PM
Ottomata triaged this task as Medium priority.May 31 2018, 4:31 PM
Ottomata set the point value for this task to 8.
Nuria raised the priority of this task from Medium to High.May 31 2018, 4:31 PM
Nuria lowered the priority of this task from High to Medium.
Nuria moved this task from Incoming to Kafka Work on the Analytics board.
Pchelolo moved this task from Backlog to watching on the Services board.Jun 4 2018, 10:05 AM
Pchelolo edited projects, added Services (watching); removed Services.
Ottomata added a comment.EditedJun 12 2018, 9:13 PM

Assuming the certificate CN is kafka_mirror_maker, we will need to add the following ACLs to clusters to which we connect over SSL port 9093:

kafka acls --add    --allow-principal User:ANONYMOUS             --topic '*' --group '*' --producer --consumer
kafka acls --add    --allow-principal User:CN=kafka_mirror_maker --topic '*' --group '*' --producer --consumer
Ottomata moved this task from Next Up to In Progress on the Analytics-Kanban board.

Change 440162 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] [WIP] SSL for Kafka MirrorMaker

https://gerrit.wikimedia.org/r/440162

Mentioned in SAL (#wikimedia-operations) [2018-06-14T17:08:16Z] <ottomata> applying ACLs to Kafka main-codfw and main-eqiad - T196081

Change 440162 merged by Ottomata:
[operations/puppet@production] SSL for Kafka MirrorMaker

https://gerrit.wikimedia.org/r/440162

Change 440378 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Enable TLS MirrorMaker consumer for main-eqiad -> main-codfw MirrorMaker

https://gerrit.wikimedia.org/r/440378

Change 440378 merged by Ottomata:
[operations/puppet@production] Enable TLS MirrorMaker consumer for main-eqiad -> main-codfw MirrorMaker

https://gerrit.wikimedia.org/r/440378

Change 440381 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Move kafka_mirror_maker ssl path to /etc/kafka/mirror/ssl

https://gerrit.wikimedia.org/r/440381

Change 440381 merged by Ottomata:
[operations/puppet@production] Move kafka_mirror_maker ssl path to /etc/kafka/mirror/ssl

https://gerrit.wikimedia.org/r/440381

Change 440384 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Enable SSL for main-codfw -> main-eqiad Kafka MirrorMaker

https://gerrit.wikimedia.org/r/440384

Change 440384 merged by Ottomata:
[operations/puppet@production] Enable SSL for main-codfw -> main-eqiad Kafka MirrorMaker

https://gerrit.wikimedia.org/r/440384

Nuria closed this task as Resolved.Jun 25 2018, 11:17 PM
Vvjjkkii renamed this task from Enable TLS and authorization for cross DC MirrorMaker to mwbaaaaaaa.Jul 1 2018, 1:07 AM
Vvjjkkii reopened this task as Open.
Vvjjkkii removed Ottomata as the assignee of this task.
Vvjjkkii raised the priority of this task from Medium to High.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed the point value for this task.
Vvjjkkii removed subscribers: gerritbot, Aklapper.
Community_Tech_bot renamed this task from mwbaaaaaaa to Enable TLS and authorization for cross DC MirrorMaker.Jul 1 2018, 6:23 AM
Community_Tech_bot closed this task as Resolved.
Community_Tech_bot assigned this task to Ottomata.
Community_Tech_bot set the point value for this task to 8.
Community_Tech_bot updated the task description. (Show Details)
CommunityTechBot lowered the priority of this task from High to Medium.Jul 3 2018, 3:24 AM