I noticed this when a CI build for Scap failed.

Indeed pip no more supports Python 2.7 since January / v 21.0. Thus we gotta pin it and instead use:

python2.7 "-m" "easy_install" "pip<21"

Test:

$ virtualenv --python python2 foo
$ cd foo
$ . ./bin/activate
(foo) $ python -m easy_install 'pip<21'
WARNING: The easy_install command is deprecated and will be removed in a future version.
Searching for pip<21
Best match: pip 20.0.2
Adding pip 20.0.2 to easy-install.pth file
Installing pip script to /tmp/foo/bin
Installing pip3.8 script to /tmp/foo/bin
Installing pip3 script to /tmp/foo/bin

Using /tmp/foo/lib/python2.7/site-packages
Processing dependencies for pip<21
Finished processing dependencies for pip<21

(foo) $ pip list
Package    Version
---------- -------
pip        20.0.2 
setuptools 44.0.0 
wheel      0.34.2

Since blubber is technically supposed to be platform agnostic, we can't assume that the base image is Debian and install any packages. Whether that's a reality I'm not sure.

Here are my thoughts. Either:

1. We drop the installation of pip entirely. If users want to use the python directives, they need to specify a base image that supplies both the python binary and pip.
2. We come up with some python version specific logic. In other words, if python.Version == "python2" { // install with pinned version of pip }, what @hashar suggested.

If the sole repository having the issue is mediawiki/tools/scap maybe we can drop the Pipeline from it and just use the tox job. It might not be worth adding python2.7 back compat code for the pipeline just for one repository?

I don't think this is something that's only related to Scap: it will affect any project using Blubber and Python. I don't know if there are others than Scap now, but there will be more in the future, I'm sure, so I think it'd make sense to change Blubber for this.

I generated a Dockerfile for Scap, using the Blubberoid service. Building the containerit results in the same problem as CI has. Changing the generated Dockerfile to use @hashar's suggestion of "<21" in the pip installation resulted in a Docker container that worked and runs the Scap unit tests etc.

There are other python projects using Blubber, so I think whatever we can do to support that is helpful.

I'm in a meeting, so I can't do this properly, but this patch to Blubber makes the Scap Docker image work for me. It prevents pip 21 for all versions of Python. I'm sure we can make it happen only for Python 2.

patch
diff --git a/config/python.go b/config/python.go
index b24b2c1..11a5c6b 100644
--- a/config/python.go
+++ b/config/python.go
@@ -110,7 +110,7 @@ func (pc PythonConfig) InstructionsForPhase(phase build.Phase) []build.Instructi
                        if pc.Requirements != nil || pc.usePoetry() {
                                if pc.Requirements != nil {
                                        ins = append(ins, build.RunAll{[]build.Run{
-                                               {pc.version(), []string{"-m", "easy_install", "pip"}},
+                                               {pc.version(), []string{"-m", "easy_install", "pip<21"}},
                                                {pc.version(), []string{"-m", "pip", "install", "-U", "setuptools", "wheel", "tox"}},
                                        }})
                                }

Change 661750 had a related patch set uploaded (by Dduvall; owner: Dduvall):
[blubber@master] python: Pin pip package to <21 for Python 2

https://gerrit.wikimedia.org/r/661750

Change 661750 merged by jenkins-bot:
[blubber@master] python: Pin pip package to <21 for Python 2

https://gerrit.wikimedia.org/r/661750