In the current arrangement, the project and tools NFS is shared with the actual database of grid engine. It is possible to remove the dependence on NFS, but without a shared storage platform, shadow master functionality becomes worse or impossible even with an external DB server.

However, this means that a malfunctioning tool is able to corrupt the gridengine database or cause the entire grid to collapse for some time even after the NFS problem is recovered.

The only thing that needs to be preserved separately really is the spooling database and related files in the /var/spool/gridengine directly. Most of toolforge will collapse if NFS is in bad enough shape anyway, but the database needs to be kept in order.

Since this would encompass only files in the .system_sge directory or even a subdirectory of that, this really could be a VM in the tools project.