The idea should be to add a few configs for librdkafka to enable TLS encryption for data in transit to Kafka-Jumbo. These are the options used for kafkatee:
kafka.security.protocol=SSL kafka.ssl.ca.location=/etc/ssl/certs/Puppet_Internal_CA.pem kafka.ssl.cipher.suites=ECDHE-ECDSA-AES256-GCM-SHA384 kafka.ssl.curves.list=P-256 kafka.ssl.sigalgs.list=ECDSA+SHA256
Plus setting port 9093 for the brokers :)
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Restricted Task | |||||
| Stalled | None | T250146 Add Authentication/Encryption to Kafka Jumbo's clients | |||
| Resolved | elukey | T250149 Enable TLS encryption from Eventgate to Kafka |
OH! I didn't realize I didn't have this enabled for eventgate-analytics! ...and it also needs enabled for eventgate-main???. I really thought I had this everywhere. Looking forward to the day when we don't have to manage 12 different values files (15 if we include eventstreams).
This might help us:
https://gerrit.wikimedia.org/r/c/operations/puppet/+/587799
Let's wait until this lands and use it and make sure we enable Kafka TLS for all event* k8s charts. I can take this one on if you like, and/or we can do it together :)
I assumed that it was not enabled, didn't check carefully!
This might help us:
https://gerrit.wikimedia.org/r/c/operations/puppet/+/587799
ack!
Let's wait until this lands and use it and make sure we enable Kafka TLS for all event* k8s charts. I can take this one on if you like, and/or we can do it together :)
+1 for doing it together!!
Change 589605 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/deployment-charts@master] eventgate - No-op use .Values.puppet_ca_cert for kafka_ca_cert file
Change 589605 merged by Ottomata:
[operations/deployment-charts@master] eventgate - No-op use .Values.puppet_ca_cert for kafka_ca_cert file
Change 589621 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/deployment-charts@master] Use private/general.yaml in event* hemlfile.yaml files
Change 589621 merged by Ottomata:
[operations/deployment-charts@master] Use private/general.yaml in event* hemlfile.yaml files
Change 589627 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/deployment-charts@master] eventgate-{main,analytics} staging - use Kafka TLS
Change 589627 merged by Ottomata:
[operations/deployment-charts@master] eventgate-{main,analytics} staging - use Kafka TLS
All staging eventgate services are now using TLS. We will do the production eventgate-main and eventgate-analytics on Monday.
Change 592658 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/deployment-charts@master] eventgate-{analytics,main} - use Kafka TLS
Change 592658 merged by Ottomata:
[operations/deployment-charts@master] eventgate-{analytics,main} - use Kafka TLS
We enabled Kafka TLS for eventgate-analytics today. We will do eventgate-main tomorrow.
Change 592951 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/deployment-charts@master] eventgate and eventstreams - Specificy kafka ssl cipher settings
Change 592951 merged by Ottomata:
[operations/deployment-charts@master] eventgate and eventstreams - Specify kafka ssl cipher settings