PodPresets were unceremoniously removed from Kubernetes in version 1.20 ( this issue is a fine gateway to the chatter https://github.com/kubernetes/website/issues/24038). Since we are approaching 1.20 as quickly as is reasonable, newer versions we deploy should not depend on that.
This is enabled via feature flag in kubeadm right now. Right now, it uses a simple label match to apply the $HOME environment variable and to mount NFS in pods. That's really handy, but we can probably accomplish it in other ways as well. A simple replacement has been created by some RedHatters https://github.com/redhat-cop/podpreset-webhook. However, since we don't absolutely need this to be a general solution for setting up arbitrary presets, we could also implement a [[ https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#mutatingadmissionwebhook | Mutating Admission Webhook ]] similar to the validating admission webhooks used for ingress and registry controls. That applies the env and volume mounts to pods that are used with certain labels in Toolforge.
Currently, the label used is `toolforge: tool`, and the preset object looks like:
```lang=yaml
apiVersion: settings.k8s.io/v1alpha1
kind: PodPreset
metadata:
name: mount-toolforge-vols
namespace: tool-<toolname>
spec:
env:
- name: HOME
value: /data/project/<toolname>
selector:
matchLabels:
toolforge: tool
volumeMounts:
- mountPath: /public/dumps
name: dumps
readOnly: true
- mountPath: /mnt/nfs/dumps-labstore1007.wikimedia.org
name: dumpsrc1
readOnly: true
- mountPath: /mnt/nfs/dumps-labstore1006.wikimedia.org
name: dumpsrc2
readOnly: true
- mountPath: /data/project
name: home
- mountPath: /etc/wmcs-project
name: wmcs-project
readOnly: true
- mountPath: /data/scratch
name: scratch
- mountPath: /etc/ldap.conf
name: etcldap-conf
readOnly: true
- mountPath: /etc/ldap.yaml
name: etcldap-yaml
readOnly: true
- mountPath: /etc/novaobserver.yaml
name: etcnovaobserver-yaml
readOnly: true
- mountPath: /var/lib/sss/pipes
name: sssd-pipes
volumes:
- hostPath:
path: /public/dumps
type: Directory
name: dumps
- hostPath:
path: /mnt/nfs/dumps-labstore1007.wikimedia.org
type: Directory
name: dumpsrc1
- hostPath:
path: /mnt/nfs/dumps-labstore1006.wikimedia.org
type: Directory
name: dumpsrc2
- hostPath:
path: /data/project
type: Directory
name: home
- hostPath:
path: /etc/wmcs-project
type: File
name: wmcs-project
- hostPath:
path: /data/scratch
type: Directory
name: scratch
- hostPath:
path: /etc/ldap.conf
type: File
name: etcldap-conf
- hostPath:
path: /etc/ldap.yaml
type: File
name: etcldap-yaml
- hostPath:
path: /etc/novaobserver.yaml
type: File
name: etcnovaobserver-yaml
- hostPath:
path: /var/lib/sss/pipes
type: Directory
name: sssd-pipes
```
Since users generally cannot create or alter these presets directly, and it is all keyed off the tool name (which is in the namespace), it's easily translated to a mutating webhook. Alternatively, the Redhat Community of Practice operator also would do the job.