@calbon Re-opening because I'd find it valuable if you could investigate this topic again for some work our team is exploring for next year. Details below, let me know what you think!

As part of our research into which moderator workflows we should prioritise next year we learned about community desire for (and existing implementations of) algorithmic reversion or prevention of obviously bad edits. This is already being done by technical volunteers on larger wikis through bots such as ClueBot NG, SeroBOT, and Salebot. The Research team has been working on T314384: Develop a ML-based service to predict reverts on Wikipedia(s), and so we could imagine leveraging this service to prevent or revert content across more projects, and in a more effective way than existing ORES-based bots.

As far as I can tell there are two avenues we could go down for this: pre-edit prevention or post-edit reversion. Post-edit reversion seems like a clear project - we could build a bot like those I linked above and plug it into an API for this model. But preventing edits before they're made might be desirable instead (or also, there are pros and cons to both approaches). AbuseFilter already has a wide range of configuration options, so this could save us development time if we could simply plug it into an existing extension rather than building something from scratch.

I've seen hesitation about leveraging AbuseFilter for this purpose because filters need to run before an edit is saved, and we're hesitant about increasing edit save times. To help us make a decision about which approach to take, I'd love to know if this is actually a blocker, or if we could imagine a system which is performant enough that we wouldn't substantially increase edit save times.

I think @calbon has looked into this far enough that we understand this is a viable approach, but we've decided to go the route of reverting edits, to increase transparency in Automoderator's decision-making.