Page MenuHomePhabricator

Use a proper dependency locking tool to define the airflow docker images python dependencies
Closed, DeclinedPublic

Description

The way we define dependencies and constraints for the Airflow docker image is too loose and ad-hoc. We should rely on a proper dependency manager/locker tool such as Poetry to lock the dependencies and export them as a requirements.txt file.

This will guarantee that any image rebuild will not silently update dependencies.

Event Timeline

brouberol triaged this task as Medium priority.
brouberol changed the task status from Open to In Progress.Aug 26 2025, 8:11 AM
brouberol claimed this task.

brouberol closed https://gitlab.wikimedia.org/repos/data-engineering/airflow/-/merge_requests/61

Replace manually tailored dependencies and constraints by a poetry lock file