Page MenuHomePhabricator

Create Bookworm-based standalone webservice image
Closed, ResolvedPublicFeature

Description

Feature summary
Please provide a bookworm-based webservice image.

Use case(s)
I use the WMF Gitlab shared CI runner to build the webservice for https://spamcheck.toolforge.org. It is written in Rust and the runners apparently use bookworm or later. This causes problems because the application is compiled against a newer glibc and thus does not even start in a Kubernetes webservice pod.

As a workaround I currently use cargo-zigbuild to build against an older glibc but that is cumbersome.

Benefits (why should this be implemented?):
Webservices based on Rust/C/C++/Zig can be built on the WMF gitlab CI runners.

Event Timeline

I'm not sure I understand this request here - the majority of the currently supported images are based on Toolforge. What image are you currently using?

The golang1.11 webservice image is on Debian buster. The jdk17 webservice image is on bullseye:

tools.spamcheck@tools-sgebastion-10:~$ cat service.template
backend: kubernetes
type: jdk17
extra_args:
  - ./run.sh
tools.spamcheck@tools-sgebastion-10:~$ kubectl exec -it spamcheck-<redacted> -- /bin/bash
tools.spamcheck@spamcheck-<redacted>:~$ cat /etc/debian_version
11.8

If there is any bookworm-based image I can use with a binary, please let me know.

taavi renamed this task from Bookworm based Toolforge Kubernetes webservice image to Create Bookworm-based standalone webservice image.Jan 17 2024, 8:33 PM
taavi added a project: cloud-services-team.
taavi triaged this task as Medium priority.

I hope most new users use https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service instead, but this is easy enough to provide after T293552: Remove Python/webservice-runner from toolforge web containers and makes sense to get people off of the anchient golang111 image.

Change 991595 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/docker-images/toollabs-images@master] Provide a standalone bookworm-web container

https://gerrit.wikimedia.org/r/991595

Unfortunately I can't use the Build Service since https://gitlab.wikimedia.org/countcount/xlinks is a monorepo for both the xlinks data collector and the spamcheck webapp. AFAICS the Multi Procfile buildpack is not supported.

AFAICS the Multi Procfile buildpack is not supported.

Is there a task for supporting that?

Is there a task for supporting that?

Not AFAIK, I created T355329.

I'm trying to understand the use cases of the different images we have. The use case for this image seems to be similar to bullseye-standalone, which comment states "Stripped down toolforge bullseye user image for binary webservices".

But I see that the bullseye-standalone image was built in a different way from the one proposed in https://gerrit.wikimedia.org/r/991595: bullseye-standalone does not inherit from bullseye-sssd, adds some extra libraries (libmariadb and libssl) and does not include the webservice-runner script. Is it actually used for cronjobs rather than webservices?

To recap, I have a few questions:

  • Do we need both bookworm-web and bookworm-standalone? For which use cases?
  • Should we include libmariadb3 and libssl1.1 in bookworm-web?
  • Do we need both bookworm-web and bookworm-standalone? For which use cases?

The toolforge-bookworm-sssd image acts as a "standalone" image (= an image where you can run a tool that doesn't need an interpreter at runtime), as it contains the base Debian layer and the SSSD config required for group permissions (and other account data loaded from LDAP) to work. The -sssd-web image in my patch builds on top of that and adds the webservice-runner shell script to run webservices in it.

  • Should we include libmariadb3 and libssl1.1 in bookworm-web?

I hope not, but I'm also not up-to-date which languages require those and which do not. Let's start without those and re-consider if there is something that requires it and can't easily use the build service.

Thanks for the clarification, I have +1'd the patch. It would be nice to document in wikitech the available images and how to choose which one to use. The only docs I could find are at https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Kubernetes#Docker_Images

The toolforge-bookworm-sssd image acts as a "standalone" image

Does it mean that anyone currently using bullseye-standalone and wishing to upgrade to Bookworm can migrate to bookworm-sssd, and we don't need bookworm-standalone at all? I remember a Phab task where the creation of the -standalone images was discussed, but I can't find it right now.

Let's start without those and re-consider if there is something that requires it

Sounds good to me.

I remember a Phab task where the creation of the -standalone images was discussed, but I can't find it right now.

The discussion was here: T277749: [Toolforge] Generic webservice not working on Kubernetes

Change 991595 merged by jenkins-bot:

[operations/docker-images/toollabs-images@master] Provide a standalone bookworm-web container

https://gerrit.wikimedia.org/r/991595

project_1317_bot_df3177307bed93c3f34e421e26c86e38 opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/192

image-config: bump to 0.0.20-20240209102849-75f6a5f8