That sensor calls an Airflow API endpoint to query a dag/task status. After webservers are migrated to Kubernetes, their API becomes Kerberos authenticated, so we need to make sure that our RestExternalTaskSensor supports it.
Description
Description
Details
Details
Related Changes in Gerrit:
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| airflow: add kerberos-related environment variables | operations/deployment-charts | master | +7 -1 |
Related Changes in GitLab:
| Title | Reference | Author | Source Branch | Dest Branch | |
|---|---|---|---|---|---|
| test_k8s: introduce a DAG querying another task status via the Airflow API | repos/data-engineering/airflow-dags!942 | brouberol | T380765 | main |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | brouberol | T362788 Migrate Airflow to the dse-k8s cluster | |||
| Resolved | brouberol | T364389 Migrate the airflow scheduler components to Kubernetes | |||
| Resolved | brouberol | T380765 Integrate Kerberos login with RestExternalTaskSensor |
Event Timeline
Comment Actions
brouberol opened https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/942
Draft: test_k8s: introduce a DAG querying another DAG status via the Airflow API
Comment Actions
Change #1098094 had a related patch set uploaded (by Brouberol; author: Brouberol):
[operations/deployment-charts@master] airflow: add kerberos-related environment variables
Comment Actions
Change #1098094 merged by Brouberol:
[operations/deployment-charts@master] airflow: add kerberos-related environment variables
Comment Actions
brouberol merged https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/942
test_k8s: introduce a DAG querying another task status via the Airflow API