Page MenuHomePhabricator

Set up new project for Zuulv3+ pre-merge and non-image-build workloads
Closed, ResolvedPublic

Description

From our meeting notes:

  • Our production Zuul executor will need to talk to this instance via ssh
  • Likely needs a Kubernetes (depending on the nodepool driver)
  • Likely to live in WMCS

Related Objects

StatusSubtypeAssignedTask
Resolvedbd808
Resolvedfnegri
Resolvedbd808
Resolvedbd808
InvalidBUG REPORTNone
In Progressbd808
ResolvedAndrew
ResolvedBUG REPORTbd808
InvalidBUG REPORTNone
OpenBUG REPORTNone
Openbd808
OpenNone
ResolvedBUG REPORTbd808
ResolvedBUG REPORTbd808
ResolvedFeaturebd808
Resolvedfnegri
OpenNone
Resolveddcaro

Event Timeline

In the 2025-06-03 collab sync we talked a bit about this and I think we decided:

  • We want a separate Cloud VPS project from the integration project that hosts the current Jenkins executors.
    • @hashar didn't think there was anything special in the config/implementation for integration that made keeping it less work than starting over.
    • The integration project will be decommissioned along with the production zuulv2 and jenkins deployments.
  • @bd808 (it's me!) volunteered to poke at using OpenTofu to drive Magnum to provision a k8s cluster.
    • This implies that he also volunteered to get somewhere setup for that to happen.
bd808 changed the task status from Open to In Progress.Jun 6 2025, 8:57 PM
bd808 claimed this task.
bd808 triaged this task as High priority.

Bikeshedding opportunity: what should we name the new Cloud VPS project? Maybe zuul-workers? Legacy Cloud VPS projects with a - in the name have a problem using the S3 storage gateway, but new projects have a UUID as the project id which eliminates that problem.

This task is theoretically blocked on T396246: Spike: Experiment with nodepool drivers for untrusted workloads, but I think we can move forward in parallel at this point.

In production it's been "as long as you don't call it zuul3" while in cloud we have an existing "zuul3" project with test instances, heh.

Do you want to separate the workers (it does sound better than executors, yes!) from those other zuul-related instances in the zuul3 project?

Do you want to separate the workers (it does sound better than executors, yes!) from those other zuul-related instances in the zuul3 project?

Yes. The T390081: Request creation of zuul3 VPS project project request was for a playground environment to run experiments. It was not intended to be a permanent home for any zuul related technology. Now that we have a long term plan to replace the https://openstack-browser.toolforge.org/project/integration project I would like us to come up with a project name that we feel we can live with for years to come. This will also give us a chance to reason about our long term needs for this project with the WMCS team.

Gotcha! Makes sense. I personally think "zuul-workers" is fine. Though "zuul-runners" would also be tad more consistent with existing "gitlab-runners" project. (and with us using the term "trusted runners" / "cloud runners" / "untrusted runners").

Gotcha! Makes sense. I personally think "zuul-workers" is fine. Though "zuul-runners" would also be tad more consistent with existing "gitlab-runners" project. (and with us using the term "trusted runners" / "cloud runners" / "untrusted runners").

+1 this was my thought, too.

zuul-runners works for me as a name for the new Cloud VPS project. Rhyming with https://openstack-browser.toolforge.org/project/gitlab-runners makes sense.

runners comes from GitLab semantic: https://docs.gitlab.com/runner/ . Zuul refers to test resources as nodes and I'd like to avoid the confusion between the two systems.

I would like to change the requested project name from zuul-runners to zuul-nodes :)

@hashar you have also asked for zuul-workers in the #wikimedia-cloud-admin IRC channel. I honestly don't care what the name is, but it would be nice to see folks participate in shared discussion rather than attempt to modify consensus without any discussion.

As you can see in this task, @Dzahn and @thcipriani both felt that alignment with gitlab-runners would actually be a benefit rather than a source of confusion. Please present your arguments to the contrary.

runners comes from GitLab semantic: https://docs.gitlab.com/runner/ . Zuul refers to test resources as nodes and I'd like to avoid the confusion between the two systems.

I would like to change the requested project name from zuul-runners to zuul-nodes :)

@hashar you have also asked for zuul-workers in the #wikimedia-cloud-admin IRC channel. I honestly don't care what the name is, but it would be nice to see folks participate in shared discussion rather than attempt to modify consensus without any discussion.

As you can see in this task, @Dzahn and @thcipriani both felt that alignment with gitlab-runners would actually be a benefit rather than a source of confusion. Please present your arguments to the contrary.

Talked about this today in the Release-Engineering-Team team meeting, we landed on zuul as the compromise. Acknowledging that there is a zuul3 project, but that project will go away beyond the proof-of-concept phase.

The project has existed for quite a while at this point. This and T396936: Provision Kubernetes cluster and bastion using OpenTofu and Magnum are starting to look like never ending tracking tasks, so let's close at least one of them.