After finishing T332031, we started seeing the below INFO stack trace on most sensors and operator logs:
[2023-03-16, 17:40:51 UTC] {logging_mixin.py:137} INFO - Exception: Traceback (most recent call last): File "/usr/lib/airflow/lib/python3.10/site-packages/datahub_provider/_plugin.py", line 281, in custom_on_success_callback datahub_task_status_callback(context, status=InstanceRunResult.SUCCESS) File "/usr/lib/airflow/lib/python3.10/site-packages/datahub_provider/_plugin.py", line 134, in datahub_task_status_callback .get_underlying_hook() File "/usr/lib/airflow/lib/python3.10/site-packages/datahub_provider/hooks/datahub.py", line 189, in get_underlying_hook conn = self.get_connection(self.datahub_conn_id) File "/usr/lib/airflow/lib/python3.10/site-packages/airflow/hooks/base.py", line 72, in get_connection conn = Connection.get_connection_from_secrets(conn_id) File "/usr/lib/airflow/lib/python3.10/site-packages/airflow/models/connection.py", line 435, in get_connection_from_secrets raise AirflowNotFoundException(f"The conn_id `{conn_id}` isn't defined") airflow.exceptions.AirflowNotFoundException: The conn_id `datahub_rest_default` isn't defined
In this task we should:
- Figure out if other Airflow instances also have this issue.
- Investigate this stack
- Figure out next steps with datahub plugin.
Note that this is cosmetic; this is just an INFO.