12:16 <yuvipanda> valhallasw: if they are large, I'd suggest: 1. force a puppet run, 2. restart them to make sure that files are in /tmp properly, 3. bind mount root to access old /tmp and clean it out
|operations/puppet : production||tools: make seperate /tmp hiera-configurable|
- Do you have a preference for when this should happen?
- Do you have any files in /tmp that need to be retained?
- Do you expect any issues with completely rebuilding the instances? e.g. the files in /tmp above.
tools-exec-gift can be rebooted when there aren't any jobs running (there is a cascade running for some days every 1st and 15th of the month). The best point of time for rebooting it now would be before 2015-05-15T00:00 (UTC). No files in /tmp. I do not expect any issues with rebuilding the instances (but you never know; I also think there is some special sge configuration (?) that would have to be retained).
@yuvipanda noted these nodes are actually all toollabs::node::compute::dedicated (as opposed to toollabs::node::compute::general, where a change was applied that needed a reboot), so we can actually safely re-enable puppet without a reboot!