Page MenuHomePhabricator

Create /shared symlink within Kubernetes images
Open, Needs TriagePublic

Description

I discovered that /shared is not available within toolforge-jobs-initiated job:

tools.urbanecmbot@tools-sgebastion-10 ~
$ toolforge-jobs run test-sleep --command 'sleep 3600' --image=tf-python39
tools.urbanecmbot@tools-sgebastion-10 ~
$ toolforge-jobs list
Job name:    Job type:    Status:
-----------  -----------  ---------
test-sleep   normal       Running
tools.urbanecmbot@tools-sgebastion-10 ~
$ kubectl exec -it test-sleep-l7j8r -- bash
tools.urbanecmbot@test-sleep-l7j8r:~$ ls /shared/pywikibot
ls: cannot access '/shared/pywikibot': No such file or directory
tools.urbanecmbot@test-sleep-l7j8r:~$ ls /shared
ls: cannot access '/shared': No such file or directory
tools.urbanecmbot@test-sleep-l7j8r:~$

/shared is usually a symbolic link to /data/project/.shared (on bastion/grid engine hosts). The target directory is normally available:

tools.urbanecmbot@test-sleep-l7j8r:~$ ls /data/project/.shared
archive  bin  cache  external-data  media  mediawiki  node  optimisations.diff  page-dumps  pagepile  piagetbot  pywikibot  pywikipedia  sqlqueries  tabletop  tcl
tools.urbanecmbot@test-sleep-l7j8r:~$

I think the /shared shortcut should be created as well, as this is what users are used to from Toolforge's normal environment. Can we do that?

Event Timeline

Change 881882 had a related patch set uploaded (by Majavah; author: Majavah):

[labs/tools/maintain-kubeusers@master] k8s_api: psp: allow mounting /shared

https://gerrit.wikimedia.org/r/881882

Change 881882 merged by jenkins-bot:

[labs/tools/maintain-kubeusers@master] k8s_api: psp: allow mounting /shared

https://gerrit.wikimedia.org/r/881882

I don't like having multiple access points for the same directory, I would prefer to decide which one is the canonical path, mark the others as "deprecated" in the docs and set a date for their removal. The shared directory is already reachable from multiple paths:

  • /data/project/shared
  • /data/project/.shared
  • /mnt/nfs/labstore-secondary-tools-project/shared/
  • /mnt/nfs/labstore-secondary-tools-project/.shared/

This wikitech page explicitly says that /data/project/shared should be used in K8s, so I don't think enabling /shared is needed, unless that's helping with the migration of some existing bots/jobs to K8s.

+1 for keeping just /data/project/shared. That one feels the cleanest for me.

I don't like having multiple access points for the same directory, I would prefer to decide which one is the canonical path, mark the others as "deprecated" in the docs and set a date for their removal.

I don't have an issue with that, but ideally, we should be consistent both within Toolforge and K8S. It took me a while to figure out why the bot does not work in k8s, but works via grid engine.

This wikitech page explicitly says that /data/project/shared should be used in K8s, so I don't think enabling /shared is needed, unless that's helping with the migration of some existing bots/jobs to K8s.

Considering the page didn't mention the /data/project/shared version at all until K8s, I think it would help with migrating existing Pywikibot-based bots using the same setup as my bots to migrate to K8S (all my bots are/were affected by this, as they were set up before the k8s job system existed). That said, if we eventually want to remove /shared everywhere, it makes sense to just do the change in those bots now (and at some point, fully remove /shared link).

taavi removed taavi as the assignee of this task.Mar 7 2023, 12:19 PM
taavi subscribed.