After adding a varnishkafka tls-enabled instance on cp1008 (cache::canary) I had an interesting chat with @BBlack about the following points:
- We clearly care about the client-side's security params because we don't want the client to be fooled into connecting to an illegitimate "broker" and feeding it sensitive data.
- We don't want to allow illegitimate clients to connect to the broker either (sending fake/confusing data, or as the output of of some kind of proxy after capturing the real client's connection).
- Neither side should allow negotiation tricks that might allow for a true end-to-end connection between our legit clients+brokers, which has bad security properties that make it more-sniffable than we'd expect.
Currently a Kafka Jumbo broker shows old/insecure algorithms/protocols when trying to connect:
CN=Puppet CA: palladium.eqiad.wmnet Client Certificate Types: RSA sign, DSA sign, ECDSA sign Requested Signature Algorithms: ECDSA+SHA512:RSA+SHA512:ECDSA+SHA384:RSA+SHA384:ECDSA+SHA256:RSA+SHA256:DSA+SHA256:ECDSA+SHA224:RSA+SHA224:DSA+SHA224:ECDSA+SHA1:RSA+SHA1:DSA+SHA1 Shared Requested Signature Algorithms: ECDSA+SHA512:RSA+SHA512:ECDSA+SHA384:RSA+SHA384:ECDSA+SHA256:RSA+SHA256:DSA+SHA256:ECDSA+SHA224:RSA+SHA224:DSA+SHA224:ECDSA+SHA1:RSA+SHA1:DSA+SHA1 Peer signing digest: SHA512 Server Temp Key: ECDH, P-256, 256 bits
Ideally, since we control both sides of the communication, we want to force clients to use TLSv2 + ECDHE-ECDSA-AES256-GCM-SHA384 (or a similar combination).
Some notes about things to test/verify:
- Additional Kafka broker TLS settings like (quoting https://docs.confluent.io/current/kafka/ssl.html):
- ssl.cipher.suites (Optional). A cipher suite is a named combination of authentication, encryption, MAC and key exchange algorithm used to negotiate the security settings for a network connection using TLS or SSL network protocol.
- ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1. It should list at least one of the protocols configured on the broker side.
- Additional librdkafka TLS settings like ssl.cipher.suites. Interestingly, librdkafka does not support any option that forces it to use a specific TLS version. Does this mean that we'd need to follow up upstream? Does ssl.cipher.suites imply a specific version of TLS?
- Are the TLS implementations used (the Open-JDK one for Kafka and OpenSSL for librdkafka) recent enough to avoid corner cases that could cause downgrades of the TLS session parameters?
- Anything else?