I checked the delays for a few emails that I get from non-WMF mailing lists, and the maximum delay that I saw was under 4 minutes, so I think that there is likely a bug on WMF's end and I'm changing the task subtype for this Phab task to "Bug Report".

I've been monitoring this the past couple days. Since yesterday we've gone from over 20k messages in the queue to less than 6k. The backlog seems to be coming from a particular provider's ratelimiting. Samples from my inbox indicate delays between when Google relays the message to the list, and delays between the list server and the outbound mail relay. The combined effect makes the delay metric you're seeing.

A large backlog might exacerbate the delays you're seeing but it is not clear if that is the cause. I will continue to monitor the situation.

Confirming that this is a serious issue. Two time-sensitive enwiki mailing lists, Oversight-L and Functionaries-en-L, do not seem to be sending out emails.

EDIT: We have confirmed that messages are being added to the Functionaries-en-L archive, even though there is no indication that they have been sent, and nobody has received them. (Oversight-L is a non-archiving list so we cannot confirm that this is occurring there as well.)

Mentioned in SAL (#wikimedia-operations) [2019-10-24T03:55:57Z] <shdubsh> temporarily turn down accept delay on fermium - T235983

This bug is also affecting the https://lists.wikimedia.org/mailman/listinfo/traffic-anomaly-report list though it is not a priority as compared to the other lists mentioned above. (Sharing in case it is helpful for debugging.)

This issue is mitigated as of this UTC morning and confirm I am no longer seeing long delays of list email.

In T235983#5603254, @ssingh wrote:

This bug is also affecting the https://lists.wikimedia.org/mailman/listinfo/traffic-anomaly-report list though it is not a priority as compared to the other lists mentioned above. (Sharing in case it is helpful for debugging.)

I can confirm this is resolved as well. Thank you.