Provide a dedicated for Abstract Wikipedia Rust image
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	cmassaro
	Nov 25 2024, 11:34 PM

Description

This image should have up-to-date versions of Rust and Cargo (>=1.81) to obviate the workarounds currently used to run Rust things in CI.

Desired behavior/Acceptance criteria (returned value, expected error, performance expectations, etc.)

Have the image agreed, made, and published
Switch function-evaluator Rust prototype over to use it

Completion checklist

Before closing this task, review one by one the checklist available here: https://www.mediawiki.org/wiki/Abstract_Wikipedia_team/Definition_of_Done#Back-end_Task/Bug_completion_checklist

Details

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	Fix name of abstract wiki rust web image	operations/docker-images/production-images	master	+1 -1
	Provide a base image for Rust, based on Bookworm using 'rustc-web' now at 1.78	operations/docker-images/production-images	master	+16 -0

Customize query in gerrit

Related Objects

Mentioned In: T400820: Use dedicated abstract-wikipeda-rust image in test-rust (and, eventually, other Rust images)

Event Timeline

cmassaro created this task.Nov 25 2024, 11:34 PM

Jdforrester-WMF added a project: serviceops.Dec 10 2024, 2:07 PM

CDanis subscribed.Dec 10 2024, 2:08 PM

Jdforrester-WMF triaged this task as Medium priority.Dec 11 2024, 5:35 PM

Jdforrester-WMF moved this task from To Triage to Engineering Backlog on the Abstract Wikipedia team board.

Joe subscribed.Dec 11 2024, 5:41 PM

hnowlan subscribed.Dec 12 2024, 12:03 PM

Change #1102983 had a related patch set uploaded (by Jforrester; author: Jforrester):

[operations/docker-images/production-images@master] Provide a base iamge for Rust 1.63, based on Bookworm

https://gerrit.wikimedia.org/r/1102983

gerritbot added a project: Patch-For-Review.Dec 12 2024, 9:51 PM

MoritzMuehlenhoff subscribed.Dec 13 2024, 1:46 PM

Jdforrester-WMF updated the task description. (Show Details)Dec 13 2024, 3:07 PM

What are the support requirements for such an image ? I 'll ask the team, but I have my doubts SRE has built any kind of significant enough knowledge to support Rust at this point in time. And it would arguably be confusing to publish such an image if we can not provide support for it.

In T380807#10405269, @akosiaris wrote:

What are the support requirements for such an image ? I 'll ask the team, but I have my doubts SRE has built any kind of significant enough knowledge to support Rust at this point in time. And it would arguably be confusing to publish such an image if we can not provide support for it.

None expected (beyond being part of the regular base-image-gets-rebuilt-every-Monday system). We intend to support this image inside the Abstract team (but defer to SRE if you want to take it over more officially, of course).

Jdforrester-WMF added a parent task: T382433: Get the Rust prototype of the function-evaluator's executors into a state where it could be deployed.Dec 18 2024, 6:24 PM

In T380807#10413489, @Jdforrester-WMF wrote:

In T380807#10405269, @akosiaris wrote:

What are the support requirements for such an image ? I 'll ask the team, but I have my doubts SRE has built any kind of significant enough knowledge to support Rust at this point in time. And it would arguably be confusing to publish such an image if we can not provide support for it.

None expected (beyond being part of the regular base-image-gets-rebuilt-every-Monday system). We intend to support this image inside the Abstract team (but defer to SRE if you want to take it over more officially, of course).

If the Abstract team intends to support that image, why not provide it also in the first place? That should avoid wrongfully set expectations to the wider org that SRE maintains that image (which at this point in time doesn't yet have the experience to do so).

In T380807#10418781, @akosiaris wrote:

In T380807#10413489, @Jdforrester-WMF wrote:

In T380807#10405269, @akosiaris wrote:

What are the support requirements for such an image ? I 'll ask the team, but I have my doubts SRE has built any kind of significant enough knowledge to support Rust at this point in time. And it would arguably be confusing to publish such an image if we can not provide support for it.

None expected (beyond being part of the regular base-image-gets-rebuilt-every-Monday system). We intend to support this image inside the Abstract team (but defer to SRE if you want to take it over more officially, of course).

If the Abstract team intends to support that image, why not provide it also in the first place? That should avoid wrongfully set expectations to the wider org that SRE maintains that image (which at this point in time doesn't yet have the experience to do so).

As discussed with your team which led to this task, this is because your team enforces the helm restriction on images used in production on k8s to be from here, and we want to use the weekly re-build infrastructure. Are you saying that that requirement can be waived?

As discussed with your team which led to this task, this is because your team enforces the helm restriction on images used in production on k8s to be from here

Not sure what "here" refers to, since we have multiple places where images can originate from, using different ways of being built, Gerrit, Gitlab, base images and production images.

What we do enforce is that the images used in production are from docker-registry.discovery.wmnet and that only trusted paths are able to publish to the registry.

All of this is that as far as I know we don't have a rule somewhere that says "don't build a Rust image in e.g. Gitlab and use it". That is still an option.

Are you saying that that requirement can be waived?

No, most definitely not. What I did wonder is why it needed to be part of the production images that SRE maintains and supports if SRE can't (as of yet at least) support it.

we want to use the weekly re-build infrastructure

This is a good technical reason, we 'll need to discuss within SRE what that would mean and how we would avoid setting false expectations of support for that image until/if we are able to support it.

This is a good technical reason, we 'll need to discuss within SRE what that would mean and how we would avoid setting false expectations of support for that image until/if we are able to support it.

Per the last AW-Security-SRE meeting, we 'll be going forward with a specifically marked for AW Rust image. The intent being to make absolutely clear to everybody else wondering whether they should be using that image that Rust is not (for now) a language supported by SRE (we don't have the capabilities for such an endeavor)

In T380807#10532025, @akosiaris wrote:

This is a good technical reason, we 'll need to discuss within SRE what that would mean and how we would avoid setting false expectations of support for that image until/if we are able to support it.

Per the last AW-Security-SRE meeting, we 'll be going forward with a specifically marked for AW Rust image. The intent being to make absolutely clear to everybody else wondering whether they should be using that image that Rust is not (for now) a language supported by SRE (we don't have the capabilities for such an endeavor)

Thanks! Done in patchset 4.

Change #1102983 merged by Alexandros Kosiaris:

[operations/docker-images/production-images@master] Provide a base image for Rust, based on Bookworm using 'rustc-web' now at 1.78

https://gerrit.wikimedia.org/r/1102983

Maintenance_bot removed a project: Patch-For-Review.Feb 15 2025, 2:30 PM

Change #1119831 had a related patch set uploaded (by Alexandros Kosiaris; author: Alexandros Kosiaris):

[operations/docker-images/production-images@master] Fix name of abstract wiki rest web image

https://gerrit.wikimedia.org/r/1119831

gerritbot added a project: Patch-For-Review.Feb 15 2025, 10:45 PM

Change #1119831 merged by Alexandros Kosiaris:

[operations/docker-images/production-images@master] Fix name of abstract wiki rust web image

https://gerrit.wikimedia.org/r/1119831

Maintenance_bot removed a project: Patch-For-Review.Feb 15 2025, 11:30 PM

The image is published and hopefully the name makes it clear that it shouldn't be reused for other purposes. We can always rename when needed.

https://docker-registry.wikimedia.org/abstractwiki-rust-web/tags/

I 'll resolve this one, feel free to reopen.

jforrester opened https://gitlab.wikimedia.org/repos/abstract-wiki/wikifunctions/function-evaluator/-/merge_requests/316

Draft: build: Switch Rust base image to new Wikimedia one

DSantamaria added a project: Essential-Work.Jul 16 2025, 1:47 PM

akosiaris renamed this task from Have SRE provide a production-ready Rust image upstream to Provide a dedicated for Abstract Wikipedia Rust image.Jul 16 2025, 1:54 PM

DSantamaria edited projects, added Abstract Wikipedia team (26Q1 (Jul–Sep)); removed Abstract Wikipedia team.Jul 16 2025, 2:07 PM

DSantamaria edited projects, added Abstract Wikipedia team; removed Abstract Wikipedia team (26Q1 (Jul–Sep)).

cmassaro mentioned this in T400820: Use dedicated abstract-wikipeda-rust image in test-rust (and, eventually, other Rust images).Jul 30 2025, 3:02 PM

jforrester merged https://gitlab.wikimedia.org/repos/abstract-wiki/wikifunctions/function-evaluator/-/merge_requests/316

build: Switch Rust base image to new Wikimedia one

Maintenance_bot removed a project: Patch-For-Review.Jul 31 2025, 5:31 PM

Jdforrester-WMF removed a parent task: T382433: Get the Rust prototype of the function-evaluator's executors into a state where it could be deployed.Oct 1 2025, 8:54 PM