At some point we've used to actively create kafka topics for Event-Platform by running ensure-topics-exist script when deploying new versions of event schemas. However, we've switched to automatic topic creation, which worked well for us in production, but apparently it doen't quite work in vagrant.
The problem is, that the topics are created automatically, but lazily, whenever the first message is produced. In production it's more-or-less OK, since we separate deployment of a consumer (change-prop) from the deployment of a producer (EventBus), and the rate of messages is pretty high, so the topic get created before changes using them in Change-Prop gets deployed. In production that works when producer and consumer are separate services.
However, in Vagrant, when the user updates after a long period of time, both new versions of a EventSchemas and Change-Prop gets started at the same time. ChangeProp then tries to subscribe to topics, that are non-existent yet (no messages were produced), which is apparently not handled well by the driver, since even when the topic is created afterwards by the same message, Change-Prop doesn't really realize that.
In production the issue also exist. If CP is both a producer and a consumer of a certain topic (like retry topic for example), deployment will be a chicken and egg problem - need to deploy and start producing and then restart for consumers to catch on.
I'm not quite sure how should we candle this issue. One idea is to revive the custom topic creation script and run it as a part of vagrant git-update command and event-schemas deployment. This might also help if we want to start supporting custom per-topic configs for T157822 or T157092
Other option is to automatically create a topic in CP if it doesn't exist, but I think it's a pretty dangerous and bad option.