Archive NFS data for projects that no longer have NFS
Closed, ResolvedPublic

Description

Lots of projects do not have NFS anymore thanks to T102240. We should keep their data archived for a few months (6 months?) but move them out of the way so they don't complicate rsyncs / etc.

yuvipanda created this task.Jul 6 2015, 2:24 PM
yuvipanda updated the task description. (Show Details)
yuvipanda raised the priority of this task from to Needs Triage.
yuvipanda added a project: Cloud-Services.
yuvipanda added a subscriber: yuvipanda.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 6 2015, 2:24 PM

Also some projects used to have /home and /project and have only one now. Those should be archived as well.

Andrew added a subscriber: Andrew.Jul 6 2015, 2:40 PM

I periodically run archive-managed-projects. From the comments there:

  1. puppet:///files/ldap/scripts/archive-managed-projects ############### #
  2. Sometimes labs projects are deleted. When this happens,
  3. orphaned volumes are left on the nfs server. ###
  4. manage-nfs-volumes-daemon records a list of of these orphans
  5. in <volumedir>/volumes-without-projects. To avoid craziness,
  6. though, it does not attempt to automatically archive them. ###
  7. This script tarballs those orphaned volumes and sets them aside for
  8. posterity. ###

So we either need to update that script to compile the list of orphans, or have the new nfs-exports-daemon maintain a list.

yuvipanda set Security to None.
Andrew claimed this task.

So the new setup should be

  1. Look at the nfs-mounts yaml file
  2. Find all the volumes that are actually active
  3. Find all folders that should be archived
  4. Archive them!

This should also be python3

Change 229458 had a related patch set uploaded (by Andrew Bogott):
Update archive-project-volumes to support our new NFS setup.

https://gerrit.wikimedia.org/r/229458

Change 229458 merged by Andrew Bogott:
Update archive-project-volumes to support our new NFS setup.

https://gerrit.wikimedia.org/r/229458

Andrew closed this task as Resolved.Aug 14 2015, 11:42 PM

The archive-project-volumes script works again, and archives are now in /srv/others/orphan-volumes.