Per today's meeting, @dcausse , @EBernhardson , @pfischer and myself discussed the resource usage of the new cirrus streaming updater. The updater will consume MWAPI, so it's important we determine the extent of cirrus streaming updater's usage of MWAPI under various scenarios (spikes, day-to-day, backfill, etc). T346456 introduced some config options for rdf-streaming-updater that should apply here as well.
Creating this ticket to:
- Estimate approximate MWAPI usage
Explore/document ways to throttle from the flink/cirrus streaming updater side- Discuss the above with ServiceOps and adjust according to their resource constraints.