Puppet has the authoritative list of Kafka brokers. Helmfiles that use Kafka hardcode that list in, and when SRE changes kafka brokers (like in T279342), helmfiles must be updated too. This is error prone and can lead to problems if SRE is not aware of what services depend on Kafka.
We attempted to use puppet to render kafka broker info into the general.yaml value file, but this doesn't quite work, because the Helm charts would have to know how to use this, and the values vary per DC.
We should see if there is a DNS or LVS based solution for this, so we don't have to hardcode lists of kafka brokers.