Kafka jumbo is currently running 1.0.0. 1.0.1 contains some bugfixes, and 1.1.0 has a couple of nice looking changes too (reduced latency for produce requests, dynamic broker configs, and improved consumer lag jmx metrics.
I'd like to upgrade main Kafka's to 1.1.0, but before doing so, we should probably upgrade jumbo first.
https://kafka.apache.org/documentation/#upgrade_1_1_0
Upgrade plan
Go to https://icinga.wikimedia.org/cgi-bin/icinga/status.cgi?search_string=Kafka+Broker+Under+Replicated+Partitions and schedule downtime for other brokers in the cluster you are working on.
- Add confluent-kafka-2.11=1.1.0 to apt.wikimedia.org
- Install confluent-kafka-2.11=1.1.0, remove unwanted systemd units and directories:
sudo apt-get install confluent-kafka-2.11 sudo rm -rv /var/log/confluent /var/lib/kafka /var/lib/zookeeper /lib/systemd/system/confluent*.service
- Restart each broker one at a time. (inter.broker.protocol.version is already set to 1.0.0.)
sudo service kafka restart
# wait until webrequest_text gets leaders for this broker again:
watch -n 5 'id=$(sudo grep broker.id= /etc/kafka/server.properties | awk -F "=" "{print \$2}"); echo "Leader: $id?"; kafka topics --describe --topic webrequest_text | grep "Leader: $id"'- Set inter.broker.protocol.version=1.1: https://gerrit.wikimedia.org/r/#/c/430398/
- restart each broker one at a time.