Set up Alerting for Data Quality dags in Airflow.
Open, Needs TriagePublic
Actions

Assigned To

Authored By

	Snwachukwu
	Oct 16 2024, 2:08 PM

Project Tags

Referenced Files

None

Subscribers

Description

We need to enable email alerting for our Deequ jobs to notify us whenever the output data is below expected value. This will require exporting our current alert table as plain text on hdfs which would be used by airflow and sent as email.

Event Timeline

Snwachukwu created this task.Oct 16 2024, 2:08 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 16 2024, 2:08 PM

Snwachukwu claimed this task.Oct 16 2024, 2:09 PM

Ahoelzl edited projects, added: Data-Engineering; removed: Data-Engineering (Q2 2024 October 1st - December 31th).Jan 18 2025, 12:27 AM

Ahoelzl moved this task from Q2 2024 October 1st - December 31th to Next Up on the Data-Engineering board.

Oh, I though we had this mechanism already.

So, today, there is no alerting whatsoever from data_quality_alerts?

CC @tchin.

Set up Alerting for Data Quality dags in Airflow.Open, Needs TriagePublicActions

Description

Event Timeline

Set up Alerting for Data Quality dags in Airflow.
Open, Needs TriagePublic
Actions