For pull-based ingestion workflows, Airflow is the recommended scheduler environment for DataHub.

It is also possible to obtain lineage relationships from the Airflow integration component.

https://datahubproject.io/docs/lineage/airflow

As we are migrating our analytics pipelines to Airflow, we should look at integrating this.

It is possible that there will be some cross-over between this ticket and T306896: Integrate Spark with DataHub with lineage (Data-Engineering) because both Airflow and Spark generate metadata within DataHub. Where we have Airflow jobs that launch Spark, we may see multiple related entries.