Page MenuHomePhabricator

Find/create proper Python >=3.7 base image for Toolhub
Closed, ResolvedPublic

Description

The issue in T274310: Building wheels for cryptography package requires rust compiler has reminded me that the base image in use by Toolhub for developing/testing/running the app was chosen because it was the only Debian Buster + Python3 base image that I could find in the Wikimedia registry at the time. I put a # FIXME: we need a non-dev Python3 buster base image comment in the file, but did not create an associated task to keep track of this tech debt.

The basic need is for a container with a Python 3.7 (or newer) runtime. Ideally this image would also include a newer pip package than the one shipped in Debian Buster, but we can probably figure out how to live with the rapidly aging pip 18.1 if there is also a 'dev' variant of the base image which includes the necessary Rust packages to support building cryptography >=3.4.

Event Timeline

Change 663345 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[operations/docker-images/production-images@master] python3: move to subdir in preparation for Buster variant

https://gerrit.wikimedia.org/r/663345

Change 663346 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[operations/docker-images/production-images@master] python3-buster: Base image for python 3.7 projects

https://gerrit.wikimedia.org/r/663346

Change 663348 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[blubber@master] python: upgrade pip before installing requirements

https://gerrit.wikimedia.org/r/663348

I found https://github.com/PyO3/pyo3/issues/1420 - it looks like they're going to make the minimum supported Rust version (MSRV) be what's currently in Buster, so apt-get install cargo -y in the build container should be good enough to get a working Rust toolchain.

bd808 moved this task from Backlog to In Progress on the Toolhub board.

Change 663592 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[wikimedia/toolhub@main] [WIP] pipeline: switch python base image

https://gerrit.wikimedia.org/r/663592

Change 663345 merged by Giuseppe Lavagetto:
[operations/docker-images/production-images@master] python3: move to subdir in preparation for Buster variant

https://gerrit.wikimedia.org/r/663345

Change 663346 merged by Giuseppe Lavagetto:
[operations/docker-images/production-images@master] python3-buster: Base image for python 3.7 projects

https://gerrit.wikimedia.org/r/663346

Successfully published image docker-registry.discovery.wmnet/python3-buster:0.0.1

Change 663348 merged by jenkins-bot:
[blubber@master] python: upgrade pip before installing requirements

https://gerrit.wikimedia.org/r/663348

Change 663592 merged by jenkins-bot:
[wikimedia/toolhub@main] pipeline: switch python base image

https://gerrit.wikimedia.org/r/663592

https://toolhub-demo.wmcloud.org/ is up and running with the new base image, so I think we can call this {{done}}.