Page MenuHomePhabricator

Explore the use of Airflow notifiers for more flexible DAG failure handling
Open, MediumPublic

Description

At the moment, the only notification of errors coming from DAGs is in the form of email.
Each instance has a default list of email addresses to which DAGs send email when tasks fail.

Airflow version 2.6 introduced the concept of notifiers, which is a more general purpose notification system that can be integrated by DAG authors.

https://airflow.apache.org/docs/apache-airflow/2.6.3/howto/notifications.html
We have now upgraded to Airflow version 2.6.3 so we have access to this feature.

I believe that this notification could be a very useful feature in terms of improving our DAG monitoring processes, since it could integrate with systems other than email.

For example, we could have DAGs send messages to Slack, or we could have DAGs integrate with Alertmanager and generate alert states with suitable notifications from that system.