As a pre-dependency for the next Kubernetes update, the cluster needs to be migrated from Pod Security Policies to Pod Security Standards.
The process is described in (feel free to extend where you see fit):
https://wikitech.wikimedia.org/wiki/Kubernetes/Clusters/PSP_replacement
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T341984 Update Kubernetes clusters to >1.25 | |||
| Stalled | JMeybohm | T273507 PodSecurityPolicies will be deprecated with Kubernetes 1.21 | |||
| Resolved | CDanis | T369491 Migrate aux cluster off of Pod Security Policies |
Change #1052700 had a related patch set uploaded (by JMeybohm; author: JMeybohm):
[operations/deployment-charts@master] aux: Add securityContext to istio components
Change #1052700 merged by Elukey:
[operations/deployment-charts@master] aux: Add securityContext to istio components
Change #1068034 had a related patch set uploaded (by Elukey; author: Elukey):
[operations/deployment-charts@master] jaeger: add securityContext configuration
Change #1068034 merged by Elukey:
[operations/deployment-charts@master] jaeger: add securityContext configuration
root@deploy1003:~# kube-env admin aux-k8s-eqiad
root@deploy1003:~# kubectl get ns -l pod-security.kubernetes.io/audit=restricted -o name | while read ns; do
kubectl label --dry-run=server --overwrite "$ns" pod-security.kubernetes.io/enforce=restricted;
done
namespace/cert-manager labeled
namespace/external-services labeled
namespace/istio-system labeled
namespace/jaeger labeled
root@deploy1003:~# kubectl get pods -A -o=jsonpath='{range .items[?(@.metadata.annotations.kubernetes\.io/psp!="privileged")]}{@.metadata.namespace}{" "}{@.metadata.annotations.kubernetes\.io/psp}{"\n"}{end}' | sort -u | column -t -s' ' | grep -v 'restricted$Change #1071132 had a related patch set uploaded (by Elukey; author: Elukey):
[operations/deployment-charts@master] admin_ng: set disablePSPMutations for AUX
Change #1071132 merged by Elukey:
[operations/deployment-charts@master] admin_ng: set disablePSPMutations for AUX
Next steps:
Change #1071872 had a related patch set uploaded (by Elukey; author: Elukey):
[operations/deployment-charts@master] aux-services: update oauth2 image for Jaeger
Change #1071872 merged by Elukey:
[operations/deployment-charts@master] aux-services: update Docker images for Jaeger
Found a violation:
annotations.authorization.k8s.io/decision allow annotations.authorization.k8s.io/reason RBAC: allowed by ClusterRoleBinding "system:controller:replicaset-controller" of ClusterRole "system:controller:replicaset-controller" to ServiceAccount "replicaset-controller/kube-system" annotations.pod-security.kubernetes.io/audit-violations would violate PodSecurity "restricted:latest": seccompProfile (pod or container "main-jaeger-agent-oauth2-sidecar" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
Change #1072156 had a related patch set uploaded (by Elukey; author: Elukey):
[operations/deployment-charts@master] jaeger: swap securityContext with podSecurityContext
Change #1072156 abandoned by Elukey:
[operations/deployment-charts@master] jaeger: swap securityContext with podSecurityContext
Reason:
sigh
Change #1072157 had a related patch set uploaded (by Elukey; author: Elukey):
[operations/deployment-charts@master] jaeger: set securityContext for the oauth sidecar
Change #1072157 merged by Elukey:
[operations/deployment-charts@master] jaeger: set securityContext for the oauth sidecar
Change #1072196 had a related patch set uploaded (by Elukey; author: Elukey):
[operations/deployment-charts@master] admin_ng: enforce PSS for the AUX cluster
Change #1072196 merged by Elukey:
[operations/deployment-charts@master] admin_ng: enforce PSS for the AUX cluster
Change #1072202 had a related patch set uploaded (by Elukey; author: Elukey):
[operations/puppet@production] kubernetes: disable PSP for the AUX cluster
Change #1072202 merged by Elukey:
[operations/puppet@production] kubernetes: disable PSP for the AUX cluster
Filed a PR for the upstream jaeger chart: https://github.com/jaegertracing/helm-charts/pull/600