In the [[https://docs.google.com/document/d/1dhAlABcM08zMcw9u01qwukhnw2bf6jQ9rKsRkuRRjdQ/edit|original design document]] for the Ceph cluster that supports the #data-platform group, only a passing reference was made to the [[https://docs.ceph.com/en/reef/cephfs/|Ceph File System]] and the Metadata Service (MDS) daemons that it requires.
> Metadata servers (MDS) - these enable a shared Posix file system interface. This shared file system is an entirely optional component of the cluster, but could be useful.
Subsequently, when working on the technical details of the cluster, we concentrated on the [[https://docs.ceph.com/en/reef/rbd/|Block Device]] and [[https://docs.ceph.com/en/reef/radosgw/|Object Gateway]] functionality. Both of these interfaces are now in active use.
However, we have now identified some use cases where the Ceph File System (`cephfs`) would be extremely useful, so we now plan to enable this additional interface and to integrate it with the dse-k8s cluster as an additional container storage interface (CSI) plugin, alongside the block device plugin.
One of the primary use cases would be to enable file systems to be provisioned to Kubernetes workloads with a `ReadWriteMany` access mode.
See: https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes for details of these access modes.
This would enable persistent volumes to be made available across the cluster, with specific components able to update the filesystem contents.
The first use cases for this functionality will be Airflow DAGs distribution (T368033) and Kerberos ticket caches (T375871).
This ticket will be an umbrella task to enable this functionality, with sub-tasks for the required elements.
We have decided to continue with the colocated design of the cluster, so the `mds` daemons will be installed to the five `cephosd100[1-5]` servers.
We will be following the guidelines for enabling cephfs from here:
[x] Create cephx authentication keys for the mds servers
[x] Deploy mds services to the cephosd cluster - https://docs.ceph.com/en/reef/cephfs/add-remove-mds/
[x] Create the pools required for cephfs - See: https://docs.ceph.com/en/reef/cephfs/createfs/#creating-pools
[] Import the upstream https://github.com/ceph/ceph-csi/tree/devel/charts/ceph-csi-cephfs and adapt to our needs
[] Ensure that the [[https://wikitech.wikimedia.org/wiki/Kubernetes/Upstream_Helm_charts_policy|Upstream helm chart policy]] is respected.
[x] Create cephx authentication keys for the ceph-csi-cephfs plugin
[] Deploy the ceph-csi-cephfs chart to the dse-k8s-cluster