PAWS currently mounts nfs into the cluster nodes, then the pods use a hostpath to mount into themselves. This prevents us from moving to a more cloud based solution, as it requires we have special worker nodes. Additionally has resulted in PAWS going down due to stale nfs mounts on some of the nodes. Should this task be completed we should be able to remove that requirement from the puppet manifest for PAWS increasing stability of the service.
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | rook | T280792 Investigate Openstack Magnum | |||
| Resolved | rook | T308873 Investigate moving PAWS to magnum | |||
| Resolved | rook | T321886 mount nfs directly into pods |
Event Timeline
Comment Actions
I am aware of one breaking change in this patch.
Currently one can access nfs through a series of sim links on the worker node as follows:
rook@paws-k8s-worker-3:~$ ls -l /public/dumps/ total 8 lrwxrwxrwx 1 root root 54 Oct 3 13:05 incr -> /mnt/nfs/dumps-clouddumps1001.wikimedia.org/other/incr lrwxrwxrwx 1 root root 70 Oct 3 13:05 pagecounts-all-sites -> /mnt/nfs/dumps-clouddumps1001.wikimedia.org/other/pagecounts-all-sites lrwxrwxrwx 1 root root 64 Oct 3 13:05 pagecounts-raw -> /mnt/nfs/dumps-clouddumps1001.wikimedia.org/other/pagecounts-raw lrwxrwxrwx 1 root root 59 Oct 3 13:05 pageviews -> /mnt/nfs/dumps-clouddumps1001.wikimedia.org/other/pageviews lrwxrwxrwx 1 root root 44 Oct 3 13:05 public -> /mnt/nfs/dumps-clouddumps1001.wikimedia.org/
It is unclear to me on why these links to other/ exist, as they are all accessible through public. As such anyone accessing data through /public/dumps/<incr/pagecounts-all-sites/pagecounts-raw/pageviews> would experience a file not found kind of error with this change. This seems manageable through an announcement of the change. Though is anyone aware of why these links exist to begin with?
Comment Actions
Mentioned in SAL (#wikimedia-cloud) [2022-11-28T15:43:49Z] <Rook> drop hostpath T321886 ce2b1f59101e5eb978c2c04d138e704f1363e9fb