This has happened three times now:
Friday 2/10, Monday 2/14, and Tuesday 2/15 from 01:13 UTC to 01:47 UTC
It not finishing its run causes antifraud to back up as seen below
https://frmon.wikimedia.org/d/Pq1YNMviz/fundraising-overview?orgId=1&viewPanel=11&from=1675987200000&to=1676505599000
Looks like it has no time limit so adding one should avoid this in the future
So, we will gain the ability to set this limit this as a result of the work that Eileen's doing on generic Queue Consumer actions. BUT.... I propose we don't add one. This has always happened as a result of our getting hit by a spammer, and it's better that we get failmail when that happens. It's of course better to detect even earlier but this is at least one way to learn about it.
@Ejegg you dropped out before we discussed but status quo now is we have set to a 11 minute time out cos we thought we would still see the fail mail - ok to change if you want tho