Page MenuHomePhabricator

Add workaround to regenerate kafka config archive on apply
Closed, ResolvedPublic5 Estimated Story Points

Description

When adding/removing kafka topics or changing kafka topic configurations, the MSK works with an archive file to apply new config. (Refer to infrastructure/services 10_msk.tf)

When the lambda function that redeploys kafka (kafka_deployer_lambda) is run, we notice that the archive file where the new config is supposed to be does not get regenerated. This is a terraform expected behaviour and documented here.

We will need to find a way to be able to regenerate the config archive with apply.

Acceptance criteria:
Topic changes get reflected when running the kafka deployer lambda.

Event Timeline

Protsack.stephan updated the task description. (Show Details)
Tim.abdullin changed the task status from Open to In Progress.May 10 2023, 2:17 PM
Tim.abdullin changed the task status from In Progress to Open.
Tim.abdullin claimed this task.
Tim.abdullin removed Tim.abdullin as the assignee of this task.
Tim.abdullin subscribed.
Tim.abdullin changed the task status from Open to In Progress.May 15 2023, 8:37 AM
Tim.abdullin claimed this task.
prabhat claimed this task.
prabhat reassigned this task from prabhat to Tim.abdullin.