In order to allow the client to catch up after a disconnect, we need to retain the events for a minimum of 4 days.
To do
- add retention to "aws.structured-data.articles.v1" topic in msk.tf
{ "config" : { "cleanup.policy" : "delete", "retention.ms" : "345,600,000 " // 4 days } }
- After 4 days of deployment, go on kafka-ui dev. From this topic, pick few events with partition, offset and ROWTIME.
- Run the wme client parallel-realtime example using offsets and then since_per_partition. Measure how long approx. it takes to fetch the first message for the partition for which we set offset/since_per_partition. Do this for few messages.
Acceptance criteria
Able to receive messages from partition with offset and since_per_partition as in the last item above.