Page MenuHomePhabricator

Investigate datahub stack trace on an-airflow1004.eqiad.wmnet
Closed, ResolvedPublic

Description

After finishing T332031, we started seeing the below INFO stack trace on most sensors and operator logs:

[2023-03-16, 17:40:51 UTC] {logging_mixin.py:137} INFO - Exception: Traceback (most recent call last):
  File "/usr/lib/airflow/lib/python3.10/site-packages/datahub_provider/_plugin.py", line 281, in custom_on_success_callback
    datahub_task_status_callback(context, status=InstanceRunResult.SUCCESS)
  File "/usr/lib/airflow/lib/python3.10/site-packages/datahub_provider/_plugin.py", line 134, in datahub_task_status_callback
    .get_underlying_hook()
  File "/usr/lib/airflow/lib/python3.10/site-packages/datahub_provider/hooks/datahub.py", line 189, in get_underlying_hook
    conn = self.get_connection(self.datahub_conn_id)
  File "/usr/lib/airflow/lib/python3.10/site-packages/airflow/hooks/base.py", line 72, in get_connection
    conn = Connection.get_connection_from_secrets(conn_id)
  File "/usr/lib/airflow/lib/python3.10/site-packages/airflow/models/connection.py", line 435, in get_connection_from_secrets
    raise AirflowNotFoundException(f"The conn_id `{conn_id}` isn't defined")
airflow.exceptions.AirflowNotFoundException: The conn_id `datahub_rest_default` isn't defined

In this task we should:

  1. Figure out if other Airflow instances also have this issue.
  2. Investigate this stack
  3. Figure out next steps with datahub plugin.

Note that this is cosmetic; this is just an INFO.

Event Timeline

Chiming in to confirm that the research instance also shows this trace too.

Seeing this in Structured Data's pipelines as well.

lbowmaker claimed this task.
lbowmaker subscribed.

Resolving, we have moved forward with different versions of Airflow since this ticket was created. Can re-open if issue re-occurs.