Page MenuHomePhabricator

Include "make" in all images
Closed, ResolvedPublic

Description

I've migrated some code from gridengine to the new kubernetes based jobs framework. The code is written in Java but there's also a Makefile to start various tasks. I've noticed that some images contain "make", others don't:

tf-ruby-27: yes
tf-perl532: yes
tf-python39: yes
tf-node16: yes

tf-php74: no
tf-jdk17: no
tf-golang: no

Would it be possible to include "make" in all toolforge images? It's a standard tool I imagine other developers might find useful too.

Details

Event Timeline

The images that have make today are quite likely the set where we are installing Debian's build-essential package:

$ git grep build-essential
bullseye0/build/Dockerfile.template:"build-essential",
node10-sssd/base/Dockerfile.template:        "build-essential",
node12-sssd/base/Dockerfile.template:        "build-essential",
node16-sssd/base/Dockerfile.template:        "build-essential",
node6-sssd/base/Dockerfile.template:        "build-essential",
perl532-sssd/base/Dockerfile.template:        "build-essential",
python2-sssd/base/Dockerfile.template:        "build-essential",
python34-sssd/base/Dockerfile.template:        "build-essential",
python35-sssd/base/Dockerfile.template:        "build-essential",
python37-sssd/base/Dockerfile.template:        "build-essential",
python39-sssd/base/Dockerfile.template:        "build-essential",
ruby21-sssd/base/Dockerfile.template:        "build-essential",
ruby25-sssd/base/Dockerfile.template:        "build-essential",
ruby27-sssd/base/Dockerfile.template:        "build-essential",
$ apt-cache depends build-essential
build-essential
 |Depends: libc6-dev
  Depends: <libc-dev>
    libc6-dev
  Depends: gcc
  Depends: g++
  Depends: make
  Depends: dpkg-dev

It would make more sense to install make directly in base than to add build-essential to all images I think.

I've been looking at submitting a patch for this myself, but while building the docker images from https://gerrit.wikimedia.org/g/operations/docker-images/toollabs-images
I get the following error:

$ ./build.py jdk17-sssd/base
...
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
##############################################################################
  Building docker-registry.tools.wmflabs.org/toolforge-jdk17-sssd/base:testing
##############################################################################
[+] Building 0.6s (3/3) FINISHED
 => [internal] load build definition from Dockerfile                                      0.0s
 => => transferring dockerfile: 507B                                                      0.0s
 => [internal] load .dockerignore                                                         0.0s
 => => transferring context: 60B                                                          0.0s
 => ERROR [internal] load metadata for docker-registry.tools.wmflabs.org/toolforge-bulls  0.5s
------
 > [internal] load metadata for docker-registry.tools.wmflabs.org/toolforge-bullseye-sssd:testing:
------
failed to solve with frontend dockerfile.v0: failed to create LLB definition: docker-registry.tools.wmflabs.org/toolforge-bullseye-sssd:testing: not found
Traceback (most recent call last):
  File "/Users/jberkel/git/toollabs-images/./build.py", line 236, in <module>
    main()
  File "/Users/jberkel/git/toollabs-images/./build.py", line 221, in main
    build_image(
  File "/Users/jberkel/git/toollabs-images/./build.py", line 113, in build_image
    subprocess.check_call(args)
  File "/Users/jberkel/.asdf/installs/python/3.10.6/lib/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/local/bin/docker', 'build', '-t', 'docker-registry.tools.wmflabs.org/toolforge-jdk17-sssd-base:testing', '/Users/jberkel/git/toollabs-images/jdk17-sssd/base']' returned non-zero exit status 1.

It works when adding -t latest.

Change 864828 had a related patch set uploaded (by Jberkel; author: Jberkel):

[operations/docker-images/toollabs-images@master] Make "make" available in all images

https://gerrit.wikimedia.org/r/864828

bd808 changed the task status from Open to In Progress.May 15 2023, 10:45 PM
bd808 assigned this task to jberkel.
bd808 updated Other Assignee, added: bd808.

Change 864828 merged by jenkins-bot:

[operations/docker-images/toollabs-images@master] Make "make" available in all images

https://gerrit.wikimedia.org/r/864828

Mentioned in SAL (#wikimedia-cloud) [2023-05-15T22:50:44Z] <bd808> Rebuilding bullseye and buster docker containers to pick up make package addition (T320343)

Thanks for moving this one forward!

$ become bd808-test -- webservice jdk17 shell -- which make
/usr/bin/make