Page MenuHomePhabricator

New instances don't always get their /data and /home project mounted from NFS
Closed, ResolvedPublic

Description

There's a race condition. It doesn't recover after additional puppet-runs as far as I know. Just requires a manual reboot of the instance.

Happens to about 3 out of 10 instances.

Event Timeline

Krinkle created this task.Mar 6 2015, 9:50 PM
Krinkle raised the priority of this task from to Needs Triage.
Krinkle updated the task description. (Show Details)
Krinkle added projects: Cloud-Services, Cloud-VPS.
Krinkle added a subscriber: Krinkle.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 6 2015, 9:50 PM
chasemp triaged this task as High priority.Nov 30 2015, 5:18 PM
chasemp added a subscriber: chasemp.

it's happened to me before as well

Krinkle removed a subscriber: Krinkle.Dec 2 2015, 1:17 PM
chasemp closed this task as Resolved.Mar 2 2016, 8:08 PM

The logic there now is showmounts waits for the NFS export to be present on initial puppet run if an instance is meant to have NFS mounts. This waits for the nfs-exports task to run on labstore1001 to fulfill async to this initial VM up operation. That server has historically been way overloaded, but with some of our recent changes we are surviving in lieu of more hardware and I am unable to reproduce this atm. I am resolving until we see it again.