dumps: true
home: true
project: true
scratch: true

How can we tell if we are using NFS or not? ;)

Well first thing would be to go through your instances and check which ones have nfs entries showing on df (they usually get mounted under /mnt/nfs/, and come from one of the labstore* servers). Then you should look at where those get mounted to, what files are in there, what applications write those files, and whether an application running on one host needs to be able to access files written to that NFS share on another host. And also whether you have anything accessing stuff like dumps.

@Addshore, can you please follow up on this? I see @https://phabricator.wikimedia.org/p/LucasWerkmeister/ creating a fair number of files in /home but not much in /data/project.

Given @Addshore is off for a few days, I'll happily take this over.

thanks @WMDE-leszek. the ask here isn't really to do anything other than document here which NFS mounts your project is using, and for what.

In T208416#5874432, @Andrew wrote:

@Addshore, can you please follow up on this? I see @https://phabricator.wikimedia.org/p/LucasWerkmeister/ creating a fair number of files in /home but not much in /data/project.

If you were referring to the ones in wikidata-dev (or was this shared between instances?), I’ve cleared that out now. I don’t think it was required anymore.

Do we need to do anything else with this task?

If the NFS isn't needed, then it'll become a task for WMCS to coordinate removing NFS from the project. Since it uses /home, that just requires a little communication and care.

Hello again @Addshore!

The ask for this remains the same as always. VMs in this project have quite a few NFS volumes mounted, including:

/data/project
/data/scratch
/mnt/nfs/dumps-labstore1006.wikimedia.org
/public/dumps
/home
/mnt/nfs/<other things>

I would like to know which of those mounts are used by your project, so that I can remove access to the unused ones. So, for example, if people have valuable files in their homedirs and/or need to be able to share homedirs between VMs, that would be a reason to continue to have an nfs mount for /home.

I'm pretty sure you could remove everything, but some people might want to rescue things from /home
Ping @Ladsgroup @Tarrow @ItamarWMDE are you using one of these mounts in the new treasure hunt project?
@Andrew is there a way to turn off the nfs mounts but copy all the home dirs onto each of the machines?
If not I can send around an email prompting people to grab what they want.

We use my home page for now for the first runs of treasure hunt (in reference-island VM only). I didn't use /srv/ first because in larger scales (hopefully in two weeks), I didn't use /srv/ and used /home/ instead because I wanted to mount to /srv/ using this standard help: https://wikitech.wikimedia.org/wiki/Help:Adding_Disk_Space_to_Cloud_VPS_instances

There's not a straightforward way to preserve the contents of /home outside of manual copying. For now I'll remove all the mounts other than that one, then you can ping if/when y'all think you're clear of it.

Thanks for the response!

Change 597413 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Remove most (but not all) nfs mounts from the wikidata-dev project.

https://gerrit.wikimedia.org/r/597413

Change 597413 merged by Andrew Bogott:
[operations/puppet@production] Remove most (but not all) nfs mounts from the wikidata-dev project.

https://gerrit.wikimedia.org/r/597413

I've removed all the mounts other than /home from all VMs in this project.

Is there a desire for us to also remove the home mount?
I don't think we need the NFS ability, but we will need to migrate some data out of some home dirs!

If you're able to muster everyone into backing up their homedirs, I'd love to remove the mount.

A few weeks is just fine. Thank you!

In T208416#6150320, @Ladsgroup wrote:

We use my home page for now for the first runs of treasure hunt (in reference-island VM only). I didn't use /srv/ first because in larger scales (hopefully in two weeks), I didn't use /srv/ and used /home/ instead because I wanted to mount to /srv/ using this standard help: https://wikitech.wikimedia.org/wiki/Help:Adding_Disk_Space_to_Cloud_VPS_instances

Did this get moved out of your home dir @Ladsgroup ?

In T208416#6154420, @Andrew wrote:

A few weeks is just fine. Thank you!

I took my time, but just sent around 1 final reminder email.
Pending the confirmation from @Ladsgroup we should be okay to move forward in the next 1-2 weeks.

In T208416#6283610, @Addshore wrote:

In T208416#6150320, @Ladsgroup wrote:

We use my home page for now for the first runs of treasure hunt (in reference-island VM only). I didn't use /srv/ first because in larger scales (hopefully in two weeks), I didn't use /srv/ and used /home/ instead because I wanted to mount to /srv/ using this standard help: https://wikitech.wikimedia.org/wiki/Help:Adding_Disk_Space_to_Cloud_VPS_instances

Did this get moved out of your home dir @Ladsgroup ?

Yup

Sounds good! lmk when everyone is ready.

I think we are ready. It has been a while now.

Change 633773 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] WMCS nfs: remove the last mounts for wikidata-dev

https://gerrit.wikimedia.org/r/633773

Change 633773 merged by Andrew Bogott:
[operations/puppet@production] WMCS nfs: remove the last mounts for wikidata-dev

https://gerrit.wikimedia.org/r/633773