| elukey | |
| Feb 6 2026, 10:36 AM |
| F72023088: Screenshot From 2026-02-13 11-21-58.png | |
| Feb 13 2026, 10:27 AM |
In order to keep compatibility with the Bullseye version of zookeeper (conf[12]xxx hosts), the Kafka upgrade working group suggests to import the Confluent Platform 7.5.x / Kafka 3.5 version in our apt repository.
We'll need to add the necessary APT config to import the new packages.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T416669 Upgrade Kafka to version 3.x | |||
| Resolved | elukey | T416670 Package Confluent Platform 7.5.x / Kafka 3.5 |
Change #1239106 had a related patch set uploaded (by Elukey; author: Elukey):
[operations/puppet@production] aptrepo: add confluent kafka 7.5 thirdparty config
Change #1239135 had a related patch set uploaded (by Elukey; author: Elukey):
[operations/puppet@production] profile::kafka::broker: support new confluent distributions
Change #1239142 had a related patch set uploaded (by Elukey; author: Elukey):
[operations/puppet@production] role::kafka::test: prepare the cluster for the Kafka upgrade
Change #1239106 merged by Elukey:
[operations/puppet@production] aptrepo: add confluent kafka 7.5 thirdparty config
Updated the confluent and confluent75 components via update/checkupdate:
root@apt1002:/srv/wikimedia# sudo -i reprepro lsbycomponent confluent-kafka confluent-kafka | 7.4.0-1 | buster-wikimedia | thirdparty/confluent7 | amd64 confluent-kafka | 7.4.0-1 | bullseye-wikimedia | thirdparty/confluent7 | amd64 confluent-kafka | 7.5.12-1 | bullseye-wikimedia | thirdparty/confluent75 | amd64 confluent-kafka | 7.5.12-1 | bookworm-wikimedia | thirdparty/confluent75 | amd64 root@apt1002:/srv/wikimedia# sudo -i reprepro lsbycomponent confluent-kafka-2.11 confluent-kafka-2.11 | 1.1.0-1 | buster-wikimedia | thirdparty/confluent | amd64, i386 confluent-kafka-2.11 | 1.1.0-1 | bullseye-wikimedia | thirdparty/confluent | amd64, i386 confluent-kafka-2.11 | 1.0.2cp2-1 | bookworm-wikimedia | thirdparty/confluent | amd64
The last one is weird, so I took a look to the deb page:
So in theory confluent-kafka-2.11 | 1.0.2cp2-1 should be a more recent version, but since I'd like to avoid surprises:
root@apt1002:/srv/wikimedia# reprepro --component thirdparty/confluent remove bookworm-wikimedia confluent-kafka-2.11 Exporting indices... Deleting files no longer referenced... root@apt1002:/srv/wikimedia# reprepro -C thirdparty/confluent copy bookworm-wikimedia bullseye-wikimedia confluent-kafka-2.11 Exporting indices... root@apt1002:/srv/wikimedia# sudo -i reprepro lsbycomponent confluent-kafka-2.11 confluent-kafka-2.11 | 1.1.0-1 | buster-wikimedia | thirdparty/confluent | amd64, i386 confluent-kafka-2.11 | 1.1.0-1 | bullseye-wikimedia | thirdparty/confluent | amd64, i386 confluent-kafka-2.11 | 1.1.0-1 | bookworm-wikimedia | thirdparty/confluent | amd64, i386
Next steps:
Change #1239135 merged by Elukey:
[operations/puppet@production] profile::kafka::broker: support new confluent distributions
Tested in pontoon, I was able to upgrade and rollback via puppet. The only detail worth to mention is that a kafka shutdown is needed before the puppet code runs, to avoid some issues with zookeeper (the broker's client tries to start a new session that is already held by another process, so it fails to startup, but a subsequent restart fixes it).
Change #1248492 had a related patch set uploaded (by Elukey; author: Elukey):
[operations/puppet@production] profile::kafka::broker: update authorizer class for kafka 3.5
Change #1248492 merged by Elukey:
[operations/puppet@production] profile::kafka::broker: update authorizer class for kafka 3.5
Change #1248496 had a related patch set uploaded (by Elukey; author: Elukey):
[operations/puppet@production] confluent: update kafka.sh with kafka-leader-election
Change #1249254 had a related patch set uploaded (by Elukey; author: Elukey):
[operations/puppet@production] aptrepo: add confluent77 component to install Kafka 3.7
Change #1249254 merged by Elukey:
[operations/puppet@production] aptrepo: add confluent77 component to install Kafka 3.7
Change #1249344 had a related patch set uploaded (by Elukey; author: Elukey):
[operations/puppet@production] profile::kafka::broker: allow to specify all 7.x confluent distros
Change #1249344 merged by Elukey:
[operations/puppet@production] profile::kafka::broker: replace Confluent 3.5 with 3.7
Change #1248496 merged by Elukey:
[operations/puppet@production] confluent: update kafka.sh with kafka-leader-election
Change #1239142 abandoned by Elukey:
[operations/puppet@production] role::kafka::test: prepare the cluster for the Kafka upgrade