PodPresets were unceremoniously removed from Kubernetes in version 1.20 ( this issue is a fine gateway to the chatter https://github.com/kubernetes/website/issues/24038). Since we are approaching 1.20 as quickly as is reasonable, newer versions we deploy should not depend on that.
This is enabled via feature flag in kubeadm right now. Right now, it uses a simple label match to apply the $HOME environment variable and to mount NFS in pods. That's really handy, but we can probably accomplish it in other ways as well. A simple replacement has been created by some RedHatters https://github.com/redhat-cop/podpreset-webhook. However, since we don't absolutely need this to be a general solution for setting up arbitrary presets, we could also implement a Mutating Admission Webhook similar to the validating admission webhooks used for ingress and registry controls. That applies the env and volume mounts to pods that are used with certain labels in Toolforge.
Currently, the label used is toolforge: tool, and the preset object looks like:
apiVersion: settings.k8s.io/v1alpha1 kind: PodPreset metadata: name: mount-toolforge-vols namespace: tool-<toolname> spec: env: - name: HOME value: /data/project/<toolname> selector: matchLabels: toolforge: tool volumeMounts: - mountPath: /public/dumps name: dumps readOnly: true - mountPath: /mnt/nfs/dumps-labstore1007.wikimedia.org name: dumpsrc1 readOnly: true - mountPath: /mnt/nfs/dumps-labstore1006.wikimedia.org name: dumpsrc2 readOnly: true - mountPath: /data/project name: home - mountPath: /etc/wmcs-project name: wmcs-project readOnly: true - mountPath: /data/scratch name: scratch - mountPath: /etc/ldap.conf name: etcldap-conf readOnly: true - mountPath: /etc/ldap.yaml name: etcldap-yaml readOnly: true - mountPath: /etc/novaobserver.yaml name: etcnovaobserver-yaml readOnly: true - mountPath: /var/lib/sss/pipes name: sssd-pipes volumes: - hostPath: path: /public/dumps type: Directory name: dumps - hostPath: path: /mnt/nfs/dumps-labstore1007.wikimedia.org type: Directory name: dumpsrc1 - hostPath: path: /mnt/nfs/dumps-labstore1006.wikimedia.org type: Directory name: dumpsrc2 - hostPath: path: /data/project type: Directory name: home - hostPath: path: /etc/wmcs-project type: File name: wmcs-project - hostPath: path: /data/scratch type: Directory name: scratch - hostPath: path: /etc/ldap.conf type: File name: etcldap-conf - hostPath: path: /etc/ldap.yaml type: File name: etcldap-yaml - hostPath: path: /etc/novaobserver.yaml type: File name: etcnovaobserver-yaml - hostPath: path: /var/lib/sss/pipes type: Directory name: sssd-pipes
Since users generally cannot create or alter these presets directly, and it is all keyed off a particular label on the pod (toolforge: tool), it's easily translated to a mutating webhook. Alternatively, the Redhat Community of Practice operator also would do the job.