Page MenuHomePhabricator

Upgrade Toolforge Kubernetes to latest 1.21
Closed, ResolvedPublic

Description

1.21 is the latest version as of writing.

Our notes: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Kubernetes/Upgrading_Kubernetes
Release notes: https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.21.md
Highlights:

  • PodSecurityPolicy deprecated, need to find a solution before 1.25
  • Kustomize updates
  • Kubelet structured logging
  • cronjobs v1 available (currently v1beta1)

Related Objects

StatusSubtypeAssignedTask
OpenNone
OpenNone
OpenNone
OpenNone
Resolvedrook
Resolvedtaavi
Resolvedrook
Resolvedtaavi
Resolvedtaavi
Resolvedtaavi
Resolvedtaavi
Resolvedtaavi
Resolvedtaavi
ResolvedBstorm
Resolvedtaavi
Resolvedtaavi
Resolvedtaavi
Resolvedtaavi
ResolvedBstorm
OpenNone
Resolvedrook
Resolvedtaavi
ResolvedBUG REPORTNone
OpenNone
Resolvedtaavi
Resolvedtaavi
OpenNone

Event Timeline

Change 738065 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] aptrepo: init kubeadm 1.21

https://gerrit.wikimedia.org/r/738065

Change 738065 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] aptrepo: init kubeadm 1.21

https://gerrit.wikimedia.org/r/738065

Mentioned in SAL (#wikimedia-operations) [2021-11-11T09:03:16Z] <arturo> pull all packages for buster-wikimedia/thirdparty/kubeadm-k8s-1-21 (T282942)

Change 738912 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] aptrepo: add k8s 1.21 to stretch too

https://gerrit.wikimedia.org/r/738912

Change 738912 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] aptrepo: add k8s 1.21 to stretch too

https://gerrit.wikimedia.org/r/738912

Mentioned in SAL (#wikimedia-operations) [2021-11-16T10:06:18Z] <arturo> updating deb packages on stretch-wikimedia/thirdparty/kubeadm-k8s-1-21 (T282942)

taavi changed the task status from Open to In Progress.Jan 11 2022, 4:31 PM
taavi claimed this task.

I'm going to have a look at this soon.

Mentioned in SAL (#wikimedia-cloud) [2022-02-04T18:53:02Z] <taavi> upgrading to kubernetes 1.21 T282942

Seeing the following in kube-scheduler logs after upgrading one toolsbeta control plane:

E0204 19:00:37.608748       1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PodDisruptionBudget: failed to list *v1.PodDisruptionBudget: the server could not find the requested resource
E0204 19:00:43.750145       1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.CSIStorageCapacity: failed to list *v1beta1.CSIStorageCapacity: the server could not find the requested resource

I presume that's because the other API servers haven't been upgraded yet and PodDisruptionBudget/v1 and CSIStorageCapacity/v1beta1 are both added in 1.21.

taavi updated Other Assignee, added: dcaro.