Page MenuHomePhabricator

Update cluster to 1.26
Closed, ResolvedPublic

Description

Currently k8s is on 1.23 we support 1.26 update to it.

Event Timeline

Doesn't appear to be fully deploying. Cluster deploys, but kube-system pods seem to have some issues. Main issue is maybe k8s-keystone-auth which is giving:

Warning  Failed     11m (x4 over 13m)     kubelet            Failed to pull image "registry.k8s.io/provider-os/k8s-keystone-auth:v1.18.0": rpc error: code = NotFound desc = failed to pull and unpack image "registry.k8s.io/provider-os/k8s-keystone-auth:v1.18.0": failed to resolve reference "registry.k8s.io/provider-os/k8s-keystone-auth:v1.18.0": registry.k8s.io/provider-os/k8s-keystone-auth:v1.18.0: not found

Maybe related to openstack upgrade? Appears to be attempting to pull the same tag as the previous install. Maybe a networking issue? Any firewall changes on the upgrade? I'm repeating the test in tf-infra-test. @Andrew do we have a ticket tracking the openstack upgrade that this would be more appropriate to report in?

Looks like the same kinds of things are happening in tf-infra-test

NAME                                             READY   STATUS             RESTARTS   AGE
pod/coredns-745687fb66-8jw96                     1/1     Running            0          21m
pod/coredns-745687fb66-gtdgc                     1/1     Running            0          21m
pod/dashboard-metrics-scraper-69b5985c9f-qrk9v   0/1     Pending            0          21m
pod/k8s-keystone-auth-8lz8g                      0/1     ImagePullBackOff   0          20m
pod/kube-dns-autoscaler-5d98bcfdd4-x6z2z         0/1     Pending            0          21m
pod/kube-flannel-ds-9pvt6                        1/1     Running            0          21m
pod/kube-flannel-ds-jb2hf                        1/1     Running            0          17m
pod/kubernetes-dashboard-7779844b49-bdc7d        0/1     Pending            0          21m
pod/magnum-metrics-server-56648f9f47-6f675       0/1     Pending            0          20m
pod/openstack-cloud-controller-manager-rpb86     0/1     ImagePullBackOff   0          21m

Looks like 1.26 isn't working anymore as some of the image tags that 1.26 wants have been removed. Upgrading to 1.27