See parent task and https://docs.google.com/document/d/17tY05WoaT_BloTzaIncR939k3hvhcVQ-E-8DBjo284E/edit# for context.
The rework of the Search Update Pipeline is planned to be implemented on top of Flink. The Data Value Stream seem to be owning this technical component, so collaboration will be required. The current plan isn't stable yet, but we envision the following needs from the Search Platform team to the Data Value Stream team:
- collaboration on defining the exact solution, taking into account contraints from both teams
- deployment and maintenance of a high availability Flink cluster
- standardized stream of Media wiki mutations (create / update / delete / ...)
AC:
- we had a kickoff meeting