From my review comment:

This seems like an OK solution for now, as I don't think the risk of breaking the distro's Python installation necessarily applies within a container filesystem.

dancy merged https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/61

Python: Set PIP_BREAK_SYSTEM_PACKAGES

error: externally-managed-environment

The reason pip warns is because people would often find a misguided recommendation to run it as root (sudo pip install) which ends up installing under /usr/local (or maybe worse /usr). That is shared between different use cases and since python only supports a single version of a package: upgrading the "system" module will certainly break another application.

The version packaged with Debian Bullseye did not emit any warning when running as non root. The packaged version automatically injects --user flag for non root user ( https://sources.debian.org/patches/python-pip/20.3.4-4%2Bdeb11u1/set_user_default.patch/ borrowed from Ubuntu) and has a Debian specific flag --system to allow installing to /usr/local (in case you make it user writable?). Debian Bullseye comes with pip 20.3.4, the major version is based upon the year so that is 2020 and is definitely obsolete).

Essentially what the Debian package did was to be friendly to users so that pip install just work and get everything installed in the user home directory instead of under /usr/local. That avoids breakage for other users and saves them from having to run sudo or wondering why they have a /usr/local: permission denied error.

And sirs, here is the beauty of open source, we can dig through the history and find out the reason upstream changed. In Debian Bookworm the above patch to inject --user has been removed thus essentially it now adheres to pip upstream package: bail out when trying to install outside of virtualenv.

python-pip (21.3.1+dfsg-1) unstable; urgency=medium
  * Drop patches git-split-ascii, set_user_default, str-version, superseded
    upstream. (Closes: #995959)

In conclusion, I think the issue pip is willing to address is to avoid sharing state between applications since upgrading a dependency for A might break B and specially NOT mess up with Debian provided packages (that is pep 0668.

two solutions

For a container image which has a single application, it is probably fine to not have a virtualenv and thus add --break-system-packages (you still most probably do NOT want to use sudo NOR run as USER root) and add --user (or give a `--root). That is probably the easiest low hanging fruit solution.

The second and nicer solution is to shift paradigm to upstream recommendation: use a virtualenv for the application and its dependencies. I have the use case for tox in releng/tox-buster for which I add to pin virtualenv/setuptools/pip to specific versions and they are installed in /usr/local but I should repurpose that to use a virtualenv:

• in its own virtualenv (virtualenv tox-venv) created with a non privileged user
• then use the virtualenv to install the wanted versions of tox/virtualenv/setuptools/pip ( tox-venv/bin/pip3 install <whatever>).
• run the app with tox-venv/bin/python and magic happens.

But it might be a bit more work.