Page MenuHomePhabricator

Make dumps accessible in PAWS
Open, LowPublic

Description

Task to define a method and policy to update said method of making dumps accessible in PAWS. At the moment PAWS uses the symbolic link defined at /public/dumps/incr to find what path dumps is and then mounts it to /public/dumps inside each user pod. This still feels a little hacky and prone to problems in the future, but might be the best way handle this until we are free of NFS.

Description was:
At the moment PAWS does mount /public/dumps in the users pods, but since that contains symlinks to the actual mounted NFS shares they are not accessible.

Event Timeline

Chicocvenancio triaged this task as Normal priority.Apr 14 2018, 6:44 PM

One option we have is to do a simple volume mount of /mnt/nfs/dumps-labstore1006.wikimedia.org and that should make the symlink work.

bd808 moved this task from Backlog to Dumps on the Data-Services board.May 10 2018, 3:25 PM
Chicocvenancio closed this task as Resolved.Jun 22 2018, 3:21 AM
Chicocvenancio claimed this task.

Done with host volume to /mnt/nfs/dumps-labstore1006.wikimedia.org. This is non ideal as any changes to that nfs mount structure will break things.

https://github.com/yuvipanda/paws/commit/4a79ac21a7f4807fec0548cdcbecb0f96e650e83

Chicocvenancio reopened this task as Open.Jun 30 2018, 6:14 PM
Chicocvenancio lowered the priority of this task from Normal to Low.
Chicocvenancio added a subscriber: chasemp.

As expected, the change to dumps-labstore1007.wikimedia.org broke this. I went ahead with a, slightly less brittle hack of reading the symlink from '/public/dumps/incr' to find what is the currently used nfs mount for dumps.

Still think this isn't ideal, @chasemp, do we have somewhere we can query for that info beyond the symlinks?

Just to clarify for casual readers: the dumps are currently accessible. (I used them successfully in this notebook earlier this month, thanks @Chicocvenancio for fixing this earlier!) I understand this ticket is now about implementing this in a more future-proof manner - should the task description be updated?

Just to clarify for casual readers: the dumps are currently accessible. (I used them successfully in this notebook earlier this month, thanks @Chicocvenancio for fixing this earlier!) I understand this ticket is now about implementing this in a more future-proof manner - should the task description be updated?

Done, thanks for the reminder.