Magnum release notes say:
Deprecate the use of ‘heat’ driver in favor of modern Kubernetes Cluster API based drivers: ‘k8s_capi_helm’ [1] or ‘k8s_cluster_api’ [2]. User still using ‘heat’ driver are encouraged to migrate to one of the abovementioned as ‘heat’ driver will be removed in the future.
The heat driver is not great, but the capi drivers require some pretty complex setup.
| Title | Reference | Author | Source Branch | Dest Branch | |
|---|---|---|---|---|---|
| Add 'magnum' service project in and default templates in codfw1dev | repos/cloud/cloud-vps/tofu-infra!328 | andrew | templates | main | |
| Add 'magnum' service project and default templates in codfw1dev | repos/cloud/cloud-vps/tofu-infra!325 | andrew | magnumtemplates | main | |
| Add 'magnum' service project and a couple default templates in codfw1dev | repos/cloud/cloud-vps/tofu-infra!324 | andrew | magnum | main | |
| magnum: add cluster-api-based coe cluster. | repos/cloud/cloud-vps/tf-infra-test!8 | andrew | magnum-cluster-api | main | |
| Add 'magnum' service project in codfw1dev | repos/cloud/cloud-vps/tofu-infra!248 | andrew | magnum | main | |
| Manage default security group in Octavia | repos/cloud/cloud-vps/tofu-infra!245 | andrew | octaviasecgroups | main |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Andrew | T393782 Investigate new Magnum drivers | |||
| Resolved | Andrew | T393783 Test (and implement?) Openstack Octavia lbaas | |||
| Resolved | Andrew | T394099 Understand Octavia network needs | |||
| Resolved | Andrew | T395864 Octavia infra followup | |||
| Invalid | Andrew | T402232 [upstream] [openstack] Fix capi-helm magnum driver to support more template options | |||
| Resolved | Andrew | T426431 debian packaging for magnum-cluster-api | |||
| Resolved | Andrew | T429164 Create two or three shared magnum templates |
Change #1175557 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):
[labs/private@master] Added dump capiservicek3s.yaml file
Change #1175557 merged by Andrew Bogott:
[labs/private@master] Added dummy capiservicek3s.yaml file
Change #1175555 merged by Andrew Bogott:
[operations/puppet@production] Magnum: use capi-helm driver in codfw1dev
Change #1175579 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):
[operations/puppet@production] Magnum: only include [heat_client] if heat is enabled
Change #1175579 merged by Andrew Bogott:
[operations/puppet@production] Magnum: only include [heat_client] if heat is enabled
Change #1175590 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):
[operations/puppet@production] Magnum: install python3-magnum-capi-helm when needed
Change #1175590 merged by Andrew Bogott:
[operations/puppet@production] Magnum: install python3-magnum-capi-helm when needed
Change #1175598 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):
[operations/puppet@production] Magnum: install helm when needed
Change #1175598 merged by Andrew Bogott:
[operations/puppet@production] Magnum: install helm when needed
Change #1175625 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):
[operations/puppet@production] Add k3s class for installing k3s on a single cloud-vps node
Change #1175626 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):
[operations/puppet@production] Add puppet class and profile to create k3s cluster-api worker for Magnum
Change #1175932 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):
[operations/puppet@production] Trivial profile for a cloud-vps-hosted chartmuseum instance
Change #1175932 merged by Andrew Bogott:
[operations/puppet@production] Trivial profile for a cloud-vps-hosted chartmuseum instance
Change #1176513 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):
[operations/puppet@production] Magnum/capi: switch to using wmf-internal image and helm repos
Change #1175625 merged by Andrew Bogott:
[operations/puppet@production] Add k3s class for installing k3s on a single cloud-vps node
Change #1175626 merged by Andrew Bogott:
[operations/puppet@production] Add puppet class and profile to create k3s cluster-api worker for Magnum
Change #1176513 merged by Andrew Bogott:
[operations/puppet@production] Magnum/capi: switch to using wmf-internal image and helm repos
Change #1179278 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):
[operations/puppet@production] magnum capi_helm: use the cloudinfra chartmuseum repo for helm charts
Change #1179278 merged by Andrew Bogott:
[operations/puppet@production] magnum capi_helm: use the cloudinfra chartmuseum repo for helm charts
The status of this is:
I am definitely the bottleneck on this task, but I just got back from vacation, am somewhat sick, and am surrounded by social rot and collapse (I live in Minnesota) so it'll be a bit before I can turn my full attention here.
...and, it turns out upstream has better support for the capi driver (not the capi-helm driver) so now I'm setting that up instead. https://stackhpc-kayobe-config.readthedocs.io/en/stackhpc-2025.1/configuration/magnum-capi.html
Change #1276821 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):
[operations/puppet@production] Magnum: switch codfw1dev from capi-helm to magnum-cluster-api driver
Change #1276821 merged by Andrew Bogott:
[operations/puppet@production] Magnum: switch codfw1dev from capi-helm to magnum-cluster-api driver
Change #1276844 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):
[operations/puppet@production] magnum cluster-api: include k8s config file on codfw1dev
Change #1276844 merged by Andrew Bogott:
[operations/puppet@production] magnum cluster-api: include k8s config file on codfw1dev
Change #1283047 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):
[operations/puppet@production] magnum: complete rename from capihelm to clusterapi
Change #1283051 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):
[operations/puppet@production] magnum: container-api to version 1.12.7 -- the 1.13.x doesn't work with capo
Change #1283047 merged by Andrew Bogott:
[operations/puppet@production] magnum: complete rename from capihelm to clusterapi
Change #1283051 merged by Andrew Bogott:
[operations/puppet@production] magnum: container-api to version 1.12.7 -- the 1.13.x doesn't work with capo
Change #1283079 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):
[operations/puppet@production] magnum-container-api: Install ORC CRD before setting up clusterctl
Change #1283079 merged by Andrew Bogott:
[operations/puppet@production] magnum-container-api: Install ORC CRD before setting up clusterctl
Change #1283834 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):
[operations/puppet@production] magnum.conf: set nova-api compatibility to v2.15
Change #1283834 merged by Andrew Bogott:
[operations/puppet@production] magnum.conf: set nova-api compatibility to v2.15
My blocker of the week turns out to have been this: https://github.com/vexxhost/magnum-cluster-api/issues/1028
Change #1288858 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):
[operations/puppet@production] Magnum: refactor to allow both magnum-cluster-api and heat driver
Change #1288858 merged by Andrew Bogott:
[operations/puppet@production] Magnum: refactor to allow both magnum-cluster-api and heat driver
Change #1288904 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):
[operations/puppet@production] Magnum: re-enable heat driver in codfw1dev
"The new CAPI driver and the old Heat driver are compatible and can both be active on the same deployment, and the decision of which driver is used for a given template depends on certain parameters inferred from the Magnum cluster template. For the new driver, these parameters are {'server_type': 'vm', 'os': 'ubuntu', 'coe': kubernetes'}. Drivers can be enabled and disabled using the disabled_drivers parameter in the [drivers] section of magnum.conf."
Change #1288904 merged by Andrew Bogott:
[operations/puppet@production] Magnum: re-enable heat driver in codfw1dev
Yeah! I just flip-flopped between the two drivers in codfw1dev a few times and it worked fine. The switch is a big opaque/accidental: basically if you tell magnum to make a cluster of ubuntu workers it uses the new driver vs. fedora workers it uses heat. I'll document once I have a better grip on this.
Time for a new update:
Things are back to pretty much working in codfw1dev. The only serious blocker to deployment in eqiad1 is the issue with docker_volume_size[0] which upstream seems to have a fix in process for. So, next steps are:
[0] https://github.com/vexxhost/magnum-cluster-api/issues/1028
Change #1295472 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):
[operations/puppet@production] magnum: remove refs to helm chart repo
Change #1295472 merged by Andrew Bogott:
[operations/puppet@production] magnum: remove refs to helm chart repo
Change #1295473 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):
[operations/puppet@production] magnum: enable magnum-cluster-api driver in eqiad1
Change #1295473 merged by Andrew Bogott:
[operations/puppet@production] magnum: enable magnum-cluster-api driver in eqiad1
This has run into a new blocker, logged upstream as https://github.com/vexxhost/magnum-cluster-api/issues/1057
We can proceed and just rely on ubuntu worker nodes but it sure would be nice to use debian instead. I'm going to give upstream a few days to respond before giving up my debian hopes.
andrew opened https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tf-infra-test/-/merge_requests/8
magnum: add cluster-api-based coe cluster.
andrew merged https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tf-infra-test/-/merge_requests/8
magnum: add cluster-api-based coe cluster.
andrew opened https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/324
Add 'magnum' service project and a couple default templates in codfw1dev
andrew closed https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/324
Add 'magnum' service project and a couple default templates in codfw1dev
andrew opened https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/325
Add 'magnum' service project and default templates in codfw1dev
andrew closed https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/325
Add 'magnum' service project and default templates in codfw1dev
andrew opened https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/328
Add 'magnum' service project in and default templates in codfw1dev
andrew closed https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/328
Add 'magnum' service project in and default templates in codfw1dev