Often folks want to build and test event based services in Cloud VPS. However, the stream of events available (only in deployment-prep) is too low for many use cases (machine learning, WDQS index updates, ElasticSearch updates, etc.). Much of the eventbus data can and is being made public anyway (via EventStreams). EventStreams is good for external use cases where folks just need to listen to events. But internal production services will consume directly from Kafka and use features like timestamp offset seeking and offset commits, and perhaps Kafka's recent exactly once transactional guarantees.

To do this, we will need a maintained Kafka cluster that can replicate specific topics from kafka-jumbo. This cluster will need to be network accessible from (all?) Cloud VPS projects.

Use cases

• T161731 WDQS reliable change stream for specific wiki
• ORES development/testing machine learning models
• ...