Airflow devenv deb package build failure
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• Stevemunene
	Oct 27 2025, 6:41 AM

Description

Following the guide on https://gitlab.wikimedia.org/repos/data-engineering/airflow-devenv#build-and-upgrade-instructions, we are running into some errors preventing the deployment of the new package.
bullseye-backports reached End OF Life in August of 2025 which gives and error of E: The repository 'http://mirrors.wikimedia.org/debian bullseye-backports Release' does not have a Release file..
Tried checking which of the packages require bullseye-backports by running the pdebuild without the backport DIST=bullseye-wikimedia pdebuild option and got

The following packages have unmet dependencies:
 pbuilder-satisfydepends-dummy : Depends: python3-poetry-core which is a virtual package and is not provided by any available package

                                 Depends: pybuild-plugin-pyproject which is a virtual package and is not provided by any available package

Unable to resolve dependencies!  Giving up...

This implies that the packages depending on this are python3-poetry-core and pybuild-plugin-pyproject.
Tried using the bookworm packages from https://packages.debian.org/search?keywords=pybuild-plugin-pyproject and https://packages.debian.org/bookworm/python3-poetry-core as an alternative with APT_USE_BUILT=yes BUILDRESULT=/home/stevemunene/pydep/pybuild-plugin-pyproject_5.20230130+deb12u1_all.deb BUILDRESULT=/home/stevemunene/pydep/python3-poetry-core_1.4.0-4_all.deb DIST=bullseye pdebuild or APT_USE_BUILT=yes BUILDRESULT=/home/stevemunene/pydep DIST=bullseye pdebuild which also resulted in the same error.

This is actively blocking release of any new airflow devenv packages and improvements

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		• Stevemunene	T405485 airflow-devenv fails to create when --name is long-ish
		Resolved		• Stevemunene	T408365 Airflow devenv deb package build failure

Event Timeline

• Stevemunene created this task.Oct 27 2025, 6:41 AM

• Stevemunene mentioned this in T405485: airflow-devenv fails to create when --name is long-ish.

• Stevemunene claimed this task.Oct 27 2025, 8:30 AM

• Stevemunene moved this task from Backlog - project to In Progress on the Data-Platform-SRE (2025.10.17 - 2025.11.07) board.

We solved this by building for bookworm which had all the prerequisite packages already abailable.

DIST=bookworm-wikimedia pdebuild
ls
airflow-devenv	airflow-devenv_0.0.15_amd64.build  airflow-devenv_0.0.15.dsc  airflow-devenv_0.0.15_source.changes  airflow-devenv_0.0.15.tar.xz

Then from the apt server import the files and add the deb to sources

stevemunene@apt1002:~$ sudo -i reprepro includedeb bullseye-wikimedia /home/stevemunene/airflow-devenv_0.0.15_all.deb
Exporting indices...
Deleting files no longer referenced...

stevemunene@apt1002:~$ sudo -i reprepro ls airflow-devenv
airflow-devenv | 0.0.15 | bullseye-wikimedia | amd64, i386, source

Then update the sources and upgrade from the cumin host

sudo cumin 'deploy*' 'apt-get update && apt-get upgrade -y airflow-devenv'

stevemunene@cumin1003:~$ sudo cumin 'deploy*' 'apt-cache policy airflow-devenv'
3 hosts will be targeted:
deploy[2002-2003].codfw.wmnet,deploy1003.eqiad.wmnet
OK to proceed on 3 hosts? Enter the number of affected hosts to confirm or "q" to quit: y
OK to proceed on 3 hosts? Enter the number of affected hosts to confirm or "q" to quit: 3
===== NODE GROUP =====                                                                                                                                                                                                                                                                                    
(2) deploy2002.codfw.wmnet,deploy1003.eqiad.wmnet                                                                                                                                                                                                                                                         
----- OUTPUT of 'apt-cache policy airflow-devenv' -----                                                                                                                                                                                                                                                   
airflow-devenv:                                                                                                                                                                                                                                                                                           
  Installed: 0.0.15                                                                                                                                                                                                                                                                                       
  Candidate: 0.0.15
  Version table:
 *** 0.0.15 1001
       1001 http://apt.wikimedia.org/wikimedia bullseye-wikimedia/main amd64 Packages
        100 /var/lib/dpkg/status

verify functionality

stevemunene@deploy2002:~$ airflow-devenv --help
Usage: airflow-devenv [OPTIONS] COMMAND [ARGS]...

  Manage ephemeral Airflow development environments

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

Commands:
  create   Provision an ephemeral airflow development environment
  destroy  Destroy the argument airflow development environment
  exec     Execute a command in one of the airflow development environment pods
  expose   Expose the airflow webserver UI to your local workstation
  list     List Airflow development environments
  logs     Tail logs for a given pod in your airflow development environment
  reauth   Re-obtain a new Kerberos ticket
  show     Show all pods of the argument airflow development environment
  yaml     Display all rendered yaml Kubernetes manifests for a development environment

Gehel moved this task from Done to Reported on the Data-Platform-SRE (2025.10.17 - 2025.11.07) board.Oct 31 2025, 8:23 AM

Airflow devenv deb package build failureClosed, ResolvedPublicActions

Description

Related ObjectsSearch...

Event Timeline

Airflow devenv deb package build failure
Closed, ResolvedPublic
Actions

Related Objects
Search...