I ran into this in train-dev today. This failure means that production mw-on-k8s will eventually run into this problem.
make -C webserver new-build-and-push-image
make[1]: Entering directory '/workspace/release/make-container-image/webserver'
make new-webserver-image fqin=traindev:5000/restricted/mediawiki-webserver:2024-04-18-190806-webserver
make[2]: Entering directory '/workspace/release/make-container-image/webserver'
docker build --pull --build-arg "http_proxy=" --build-arg "https_proxy=" -f Dockerfile.webserver-base-image -t webserver-image-base .
DEPRECATED: The legacy builder is deprecated and will be removed in a future release.
Install the buildx component to build images with BuildKit:
https://docs.docker.com/go/buildx/
Sending build context to Docker daemon 10.75kB
Step 1/6 : FROM docker-registry.wikimedia.org/mediawiki-httpd:latest
latest: Pulling from mediawiki-httpd
b9404085160d: Already exists
076ae664f7a3: Pulling fs layer
6a76dcc301b8: Pulling fs layer
215f82b0ccf9: Pulling fs layer
7fd81a6ad52f: Pulling fs layer
6358cde9192b: Pulling fs layer
a700ad61e889: Pulling fs layer
26aaace68329: Pulling fs layer
7b011c61239d: Pulling fs layer
6358cde9192b: Waiting
a700ad61e889: Waiting
26aaace68329: Waiting
7b011c61239d: Waiting
7fd81a6ad52f: Waiting
215f82b0ccf9: Verifying Checksum
215f82b0ccf9: Download complete
6a76dcc301b8: Verifying Checksum
6a76dcc301b8: Download complete
7fd81a6ad52f: Download complete
6358cde9192b: Download complete
26aaace68329: Verifying Checksum
26aaace68329: Download complete
7b011c61239d: Verifying Checksum
7b011c61239d: Download complete
a700ad61e889: Verifying Checksum
a700ad61e889: Download complete
...
7b011c61239d: Pull complete
Digest: sha256:1010e18858785c8bf9dedca12ef098c4496b8ce8e11c3ec0fe9cd8acce93ba39
Status: Downloaded newer image for docker-registry.wikimedia.org/mediawiki-httpd:latest
---> 9d4bb5f26cdb
Step 2/6 : ENV DEBIAN_FRONTEND=noninteractive
---> Running in 88c58ee8f167
---> Removed intermediate container 88c58ee8f167
---> 553f2f302f88
Step 3/6 : USER root
---> Running in 42aa54465fbc
---> Removed intermediate container 42aa54465fbc
---> 430643cc2480
Step 4/6 : RUN apt-get update && apt-get install -y rsync
---> Running in 431c40dd292f
Get:1 http://security.debian.org buster/updates InRelease [34.8 kB]
Get:2 http://mirrors.wikimedia.org/debian buster InRelease [122 kB]
Get:3 http://apt.wikimedia.org/wikimedia buster-wikimedia InRelease [178 kB]
Get:4 http://security.debian.org buster/updates/main amd64 Packages [595 kB]
Get:5 http://mirrors.wikimedia.org/debian buster-updates InRelease [56.6 kB]
Ign:6 http://mirrors.wikimedia.org/debian buster-backports InRelease
Err:7 http://mirrors.wikimedia.org/debian buster-backports Release
404 Not Found [IP: 208.80.154.139 80]
Get:8 http://mirrors.wikimedia.org/debian buster/main amd64 Packages [7909 kB]
Get:9 http://apt.wikimedia.org/wikimedia buster-wikimedia/main amd64 Packages [94.4 kB]
Get:10 http://mirrors.wikimedia.org/debian buster-updates/main amd64 Packages [8788 B]
Reading package lists...
E: The repository 'http://mirrors.wikimedia.org/debian buster-backports Release' does not have a Release file.
The command '/bin/sh -c apt-get update && apt-get install -y rsync' returned a non-zero code: 100
make[2]: *** [Makefile:42: webserver-base-image] Error 100
make[2]: Leaving directory '/workspace/release/make-container-image/webserver'
make[1]: *** [Makefile:33: new-build-and-push-image] Error 2
make[1]: Leaving directory '/workspace/release/make-container-image/webserver'
make: *** [Makefile:38: build-and-push-webserver-image] Error 2
19:08:15 Finished build-and-push-container-images (duration: 06m 08s)
19:08:15 Build of K8s images failed (non-K8s deployment will continue normally)This error is not happening in production yet due to caching, but it's a ticking timebomb:
... make[2]: Entering directory '/srv/mwbuilder/release/make-container-image/webserver' docker build --pull --build-arg "http_proxy=http://webproxy.eqiad.wmnet:8080" --build-arg "https_proxy=http://webproxy.eqiad.wmnet:8080" -f Dockerfile.webserver-base-image -t webserver-image-base . Sending build context to Docker daemon 10.75kB^M^M Step 1/6 : FROM docker-registry.wikimedia.org/mediawiki-httpd:latest latest: Pulling from mediawiki-httpd Digest: sha256:1010e18858785c8bf9dedca12ef098c4496b8ce8e11c3ec0fe9cd8acce93ba39 Status: Image is up to date for docker-registry.wikimedia.org/mediawiki-httpd:latest ---> 9d4bb5f26cdb Step 2/6 : ENV DEBIAN_FRONTEND=noninteractive ---> Using cache ---> baeb0c6ada71 Step 3/6 : USER root ---> Using cache ---> 886f6adfc705 Step 4/6 : RUN apt-get update && apt-get install -y rsync ---> Using cache ---> e603821309d0 Step 5/6 : RUN mkdir /srv/mediawiki && chown www-data: /srv/mediawiki ...