ChangeProp currently expands backlinks eagerly into its constituent jobs. This is nicely modular, and conceptually simple. However, since backlink expansion is fairly quick, it also causes large jobs to be processed in large batches. Consequences are:
- Large backlink jobs are fully processed before starting the processing of smaller, later jobs. This causes large delays on those smaller jobs.
- Especially expensive backlink jobs (for example, involving an expensive template) are processed bunched in time. While concurrency limits in ChangeProp are designed to keep the load constant anyway, defective timeout logic / resource limiting (ex: T97192) can lead to higher actual concurrency and resource consumption by piling up zombie requests and triggering retries in services like Parsoid.
Proposed change
Since backlinks are expanded one batch request at a time, we can increase interleaving of jobs by performing the expansion lazily, as part of the "leaf" event processing.
Current flow:
Edit event topic -> Eager expansion into leaf job topic, using continuation topic -> Leaf job execution
Proposed flow:
Edit event
-> Expand one batch to leaf job topic & add a special continuation event in the same topic
-> process leaf events
-> when encountering continuation event, continue batch expansion & add continuation if backlinks remaining.The biggest change is interleaving continuations / expansions with the leaf event processing.