When adding/removing kafka topics or changing kafka topic configurations, the MSK works with an archive file to apply new config. (Refer to infrastructure/services 10_msk.tf)
When the lambda function that redeploys kafka (kafka_deployer_lambda) is run, we notice that the archive file where the new config is supposed to be does not get regenerated. This is a terraform expected behaviour and documented here.
We will need to find a way to be able to regenerate the config archive with apply.
Acceptance criteria:
Topic changes get reflected when running the kafka deployer lambda.