TLS certificates issued by the Kafka intermediate CA expire after 4 weeks (by default), and in the future this figure may change. Manual actions will be painful for SREs, and kafka seems to have a way to reload its keystores:
elukey@kafka-test1006:~$ kafka-configs --bootstrap-server $(hostname -f):9092 --entity-name 1006 --entity-type brokers --add-config listener.name.SSL.ssl.keystore.location=/etc/kafka/ssl/kafka_test-eqiad_broker.keystore.p12 --alter Completed updating config for broker: 1006.
(Please note: I am using port 9092 since with 9093 extra ssl properties are needed, otherwise the CLI issues a out of memory exception).
More info from a similar environment: https://forge.softwareheritage.org/D5864
The idea is to either:
- Create an exec in puppet, triggered when the new keystore is created by puppet (by the code that interfaces with the PKI intermediate)
- Add a Reload override to the Kafka systemd unit in puppet to execute the command or a script like the above