The Kafka test cluster is currently not handling much traffic, and it would be nice to have some clients using it to better test the upcoming upgrade procedure.
The goal of this task is to identify 2/3 clients that produce/consume data regularly from/to Kafka test.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T416669 Upgrade Kafka to version 3.x | |||
| Resolved | None | T417034 Add some kafka clients to the Kafka test cluster |
These applications were tested during the Kafka upgrade to 3.7:
Both applications were processing data during the upgrade, there were no interruptions. Dashboard 1 - Dashboard 2
After the upgrade we also run a Gobblin job from a Stat Machine, connected to the Test cluster, and it moved around 400k records to HDFS without issues.
This has been handled out of band, and is no longer necessary to keep open now that we're performing the upgrade (or have done so, depending on the cluster).