Search platform uses Flink for WCQS and WDQS, and will soon begin using it for the search update pipeline as well.
To increase stability and security for the above services, we'd like to run Flink Operator in Kubernetes. Additionally, the Event Platform team has expressed interest in Flink operator.
Creating this task to:
- Gather requirements for Flink Operator (compute resources, k8s version, permissions inside the cluster, capabilities of limiting the blast radius of it etc)
- Deploy on DSE cluster or another appropriate evaluation environment.
- Prototype/Replicate rdf-streaming-updater deployment using the operator