Investigate setting up alerts for Growth dashboards
Closed, DuplicatePublic
Actions

Assigned To

None

Authored By

	• Tgr
	Nov 25 2020, 12:52 AM

Description

The Growth team has two dashboards currently:

a Grafana board for suggested edits task recommendation performance
a Logstash board for PHP errors originating from our components

(There's also the shared client-side error board, but filtering it by component is not trivial. That's tracked in T266522: Set up Javascript error dashboard for Growth team.)

We should consider setting up automatic alerts for them (especially for the Grafana one, the other is looked at way more often).

EventLogging data could also use alerts, to detect issues like T269966: HomepageVisit schema validation errors; not sure if there is tooling readily available for this.

Related Objects
Search...

Status	Assigned	Task
Duplicate	None	T268700 Investigate setting up alerts for Growth dashboards
Open	None	T345202 Implement alerting for Growth-consumed or Growth-managed services/pipelines
Open	None	T341710 Alert the Growth team when linkrecommendation service is unavailable
Open	None	T328129 Implement alerting when spikes occur in Growth team dashboard in Logstash
Resolved	None	T345204 Alert the Growth team when number of available task recommendations drops significantly
Open	None	T367211 Log unactionable errors to statslib/prometheus and set alert instead of using logstash
Resolved	Cyndymediawiksim	T386116 Create alerts for "Add (Section-) Image" task being broken for users ("No suggestions available")
Open	None	T388381 Add alerts for Link Recommendation task being broken for users

Event Timeline

• Tgr created this task.Nov 25 2020, 12:52 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 25 2020, 12:52 AM

In T240201#6605879, @Gilles wrote:

It might be useful to set up some alerts on that dashboard, to be notified about incidents like this in the feature without having to keep on eye on the dashboard regularly. We can show you how our Grafana-based alerts currently work.

In T240201#6623662, @Gilles wrote:

@kostajh there is: https://wikitech.wikimedia.org/wiki/Grafana.wikimedia.org#Alerts_(with_notifications_via_Icinga)

• Tgr mentioned this in T267216: Slow load times for Special:Homepage on cswiki.Nov 25 2020, 1:04 AM

kostajh awarded a token.Nov 25 2020, 9:08 AM

• Tgr mentioned this in T269966: HomepageVisit schema validation errors.Dec 16 2020, 1:53 AM

• Tgr updated the task description. (Show Details)

MBinder_WMF added a project: Growth-Team-Filtering.Apr 15 2021, 6:47 PM

Michael subscribed.Mar 10 2025, 8:07 AM

Maybe let's make this into an epic. I'll be adding subtasks. But maybe this should be tackled after T383874: Create grafana dashboards using Prometheus is done.

Relevant prior work:

alertmanager: route Growth team alerts and search: add alerts for weighted_tags indexing throughput to make alerts show up in team slack

Michael added a subtask: T345202: Implement alerting for Growth-consumed or Growth-managed services/pipelines.Mar 10 2025, 10:43 AM

In retrospect, maybe let's just make this a duplicate of T345202: Implement alerting for Growth-consumed or Growth-managed services/pipelines instead.

Michael closed this task as a duplicate of T345202: Implement alerting for Growth-consumed or Growth-managed services/pipelines.Mar 10 2025, 10:45 AM

Investigate setting up alerts for Growth dashboardsClosed, DuplicatePublicActions

Description

Related ObjectsSearch...

Event Timeline

Investigate setting up alerts for Growth dashboards
Closed, DuplicatePublic
Actions

Related Objects
Search...