Page MenuHomePhabricator

Manage provisioning and configuration of Jenkins agents using OpenTofu
Closed, DeclinedPublicFeature

Description

While working on T389554: Add some integration executors to spread the load and manually running through the checklist at https://wikitech.wikimedia.org/wiki/Nova_Resource:Integration/Setup some of us started wondering about using the relatively new https://wikitech.wikimedia.org/wiki/Help:Using_OpenTofu_on_Cloud_VPS features to manage our pool of Jenkins agents.

https://gitlab.wikimedia.org/cloudvps-repos/deployment-prep/tofu-provisioning could be inspiration for getting a basic OpenTofu setup going. More about that project and the steps it took to get started can be found in T372498: Figure out how to provision a Kubernetes cluster using Magnum and OpenTofu.

Research is needed to determine if there is an opentofu/terraform provider that can be used to configure Jenkins to work with a new node or if we would need to invent something new for that work. Even if we are not able to fully automate joining nodes to the Jenkins instance having the multi-step provisioning process reduced to changing some instance counts and flavors in a config file would be a large improvement over the current very manual process.

Event Timeline

The runbook can probably be simplified a bit:

  • drop the local Puppetserver, I don't think it is needed anymore now that the agents are rather dumb and we have access to +2er via Cloud Services and SRE collab.
  • Some of the Puppet config could be applied via a puppet prefix in Horizon
  • For adding the agents in the Jenkins controller we considered using configuration as code plugin to manage the instance configuration (T328920), not sure whether it could manage just a subset of the conf (ie just the agents). Or that could potentially be done via Puppet.

I am not sure it is worth the investment given the Continuous-Integration-Infrastructure (Zuul upgrade) we lead to decommission those agents.

I am not sure it is worth the investment given the Continuous-Integration-Infrastructure (Zuul upgrade) we lead to decommission those agents.

:nod: When I wrote this I don't think I understood how soon we were hoping/planning to do the Zuul migration. Maybe a better goal would be to make OpenTofu a core part of any of that next generation system which ends up on Cloud VPS.

I am declining using OpenTOFU to manage the Jenkins agents.

As part of the Zuul upgrade, there is work ongoing to use OpenTOFU to provide the resources for a Kubernetes cluster dedicated to Zuul: T396936