Page MenuHomePhabricator

Email alerts from Grafana stopped working?
Open, In Progress, Needs TriagePublic

Description

Wanted to check if the email alerts in Grafana been turned off ? Looking at https://grafana-rw.wikimedia.org/alerting/notifications all delivery attempts fail for all emails. It says smtp not configured. Or is it a bug?

This used to work (web teams use them for performance alerts, the last alerts was delivered the 15th of September for them) and I wanted to use it for Test Platform alerts.

Example of what it looks like:

failing.png (280×3 px, 84 KB)

Event Timeline

Bugreporter2 renamed this task from Email alerts from Grafana stopped to work? to Email alerts from Grafana stopped working?.Oct 8 2025, 10:28 AM

Hi @andrea.denisse wanted to check if you know happened here, is it an upgrade that changed it? Please let me know if there's anything I can do to help!

andrea.denisse changed the task status from Open to In Progress.EditedWed, Oct 15, 3:06 PM

Hi @andrea.denisse wanted to check if you know happened here, is it an upgrade that changed it? Please let me know if there's anything I can do to help!

Hi Peter, I'm looking into the issue.

Meanwhile, I was wondering if the recently implemented Slack alerts bot would be useful for your team as to receive those alerts. With it, your team could receive alerts for any Slack channel of your choice.

You can see an example of it on the Slack #api-alerts channel. I'll update here about the emails issue.

#api-alerts channel

Ah cool, didn't know, I'll try that out, thank you!

#api-alerts channel

Ah cool, didn't know, I'll try that out, thank you!

I noticed those alerts arrived to a Slack channel, I'm working on a patch right now.

Change #1196533 had a related patch set uploaded (by Andrea Denisse; author: Andrea Denisse):

[operations/puppet@production] alertmanager: Add Slack route for the rweb team

https://gerrit.wikimedia.org/r/1196533

Great! So from the chat: we have #performance-alerts and then #test-platform-tools-reporting. The test-platform-tools-reporting is the channel where the Test Platform team send their alerts.

Change #1196533 merged by Andrea Denisse:

[operations/puppet@production] alertmanager: Add Slack route for the rweb team

https://gerrit.wikimedia.org/r/1196533