Change Details

Hi folks, opening this task to discuss if it is worth or not to start testing a migration path to Kafka 2.x (like 2.8). As Andrew pointed out several times, the 2.x versions don't have a lot of new cool features on the broker side, since most of the work has been done on things like Kafka streams etc.. There are some good things though that I tracked over time: - Support for Java 11 (https://issues.apache.org/jira/browse/KAFKA-7264) - Support for TLS 1.3 (https://issues.apache.org/jira/browse/KAFKA-7251) - Support for dynamic reload of TLS keystores in place (https://issues.apache.org/jira/browse/KAFKA-7429) - A better and more stable Kafka Mirror Maker (even if after all the work that Andrew did we have something very stable as well now) - Initial support for Kraft (to remove the Zookeeper dependency - even if still not recommented for production) - Fix offset retention on idle streams https://issues.apache.org/jira/browse/KAFKA-4682 - [[ https://cwiki.apache.org/confluence/display/KAFKA/KIP-302+-+Enable+Kafka+clients+to+use+all+DNS+resolved+IP+addresses | KIP-302 ]] - Possibility to use a single DNS address for a Kafka cluster, instead of a static list of bootstrap server host names. Would help solve: {T253058} and {T213561} Kafka 3.1 is about to be released, so jumping to something like Kafka 2.8 could be nice to avoid lagging too much from upstream. From the [[ https://kafka.apache.org/20/documentation/streams/upgrade-guide | upgrade guide ]] the rolling upgrade seems doable/possible, maybe we could try it on the test cluster to see how it goes during the coming months.

Hi folks, opening this task to discuss if it is worth or not to start testing a migration path to a newer Kafka version. As Andrew pointed out several times, the 2.x versions don't have a lot of new cool features on the broker side, since most of the work has been done on things like Kafka streams etc.. There are some good things though that I tracked over time: - Support for Java 11 (https://issues.apache.org/jira/browse/KAFKA-7264) - Support for TLS 1.3 (https://issues.apache.org/jira/browse/KAFKA-7251) - Support for dynamic reload of TLS keystores in place (https://issues.apache.org/jira/browse/KAFKA-7429) - A better and more stable Kafka Mirror Maker (even if after all the work that Andrew did we have something very stable as well now) - Initial support for Kraft (to remove the Zookeeper dependency - even if still not recommented for production) - Fix offset retention on idle streams https://issues.apache.org/jira/browse/KAFKA-4682 - [[ https://cwiki.apache.org/confluence/display/KAFKA/KIP-302+-+Enable+Kafka+clients+to+use+all+DNS+resolved+IP+addresses | KIP-302 ]] - Possibility to use a single DNS address for a Kafka cluster, instead of a static list of bootstrap server host names. Would help solve: {T253058} and {T213561} From the [[ https://kafka.apache.org/20/documentation/streams/upgrade-guide | upgrade guide ]] the rolling upgrade seems doable/possible, maybe we could try it on the test cluster to see how it goes during the coming months.

Hi folks, opening this task to discuss if it is worth or not to start testing a migration path to Kafka 2.x (like 2.8)a newer Kafka version. As Andrew pointed out several times, the 2.x versions don't have a lot of new cool features on the broker side, since most of the work has been done on things like Kafka streams etc.. There are some good things though that I tracked over time: - Support for Java 11 (https://issues.apache.org/jira/browse/KAFKA-7264) - Support for TLS 1.3 (https://issues.apache.org/jira/browse/KAFKA-7251) - Support for dynamic reload of TLS keystores in place (https://issues.apache.org/jira/browse/KAFKA-7429) - A better and more stable Kafka Mirror Maker (even if after all the work that Andrew did we have something very stable as well now) - Initial support for Kraft (to remove the Zookeeper dependency - even if still not recommented for production) - Fix offset retention on idle streams https://issues.apache.org/jira/browse/KAFKA-4682 - [[ https://cwiki.apache.org/confluence/display/KAFKA/KIP-302+-+Enable+Kafka+clients+to+use+all+DNS+resolved+IP+addresses | KIP-302 ]] - Possibility to use a single DNS address for a Kafka cluster, instead of a static list of bootstrap server host names. Would help solve: {T253058} and {T213561} Kafka 3.1 is about to be released, so jumping to something like Kafka 2.8 could be nice to avoid lagging too much from upstream. From the [[ https://kafka.apache.org/20/documentation/streams/upgrade-guide | upgrade guide ]] the rolling upgrade seems doable/possible, maybe we could try it on the test cluster to see how it goes during the coming months.