Page MenuHomePhabricator

New instance fails to mount /home
Closed, ResolvedPublic

Description

Tried creating a new instance for catgraph. NFS mount /home isn't mounted. Relevant lines from syslog:

Jan 27 19:12:10 fridolin puppet-agent[15810]: Sleeping for 17 seconds (splay is enabled)
Jan 27 19:12:28 fridolin puppet-agent[15810]: Retrieving pluginfacts
Jan 27 19:12:28 fridolin puppet-agent[15810]: Retrieving plugin
Jan 27 19:12:28 fridolin puppet-agent[15810]: Loading facts
Jan 27 19:12:33 fridolin puppet-agent[15810]: Caching catalog for fridolin.catgraph.eqiad.wmflabs
Jan 27 19:12:34 fridolin puppet-agent[15810]: Applying configuration version '1453921938'
Jan 27 19:12:34 fridolin crontab[16008]: (root) LIST (root)
Jan 27 19:15:01 fridolin CRON[17610]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Jan 27 19:15:41 fridolin puppet-agent[15810]: /usr/local/sbin/block-for-export labstore.svc.eqiad.wmnet project/catgraph 180 returned 2 instead of one of [0]
Jan 27 19:15:41 fridolin puppet-agent[15810]: (/Stage[main]/Role::Labs::Instance/Exec[block-for-project-export]/returns) change from notrun to 0 failed: /usr/local/sbin/block-for-export labstore.svc.eqiad.wmnet project/catgraph 180 returned 2 instead of one of [0]
Jan 27 19:15:41 fridolin puppet-agent[15810]: (/Stage[main]/Role::Labs::Instance/Mount[/data/project]) Dependency Exec[block-for-project-export] has failures: true
Jan 27 19:15:41 fridolin puppet-agent[15810]: (/Stage[main]/Role::Labs::Instance/Mount[/data/project]) Skipping because of failed dependencies
Jan 27 19:17:01 fridolin CRON[18716]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Jan 27 19:18:44 fridolin puppet-agent[15810]: /usr/local/sbin/block-for-export labstore.svc.eqiad.wmnet project/catgraph 180 returned 2 instead of one of [0]
Jan 27 19:18:44 fridolin puppet-agent[15810]: (/Stage[main]/Role::Labs::Instance/Exec[block-for-home-export]/returns) change from notrun to 0 failed: /usr/local/sbin/block-for-export labstore.svc.eqiad.wmnet project/catgraph 180 returned 2 instead of one of [0]
Jan 27 19:18:44 fridolin puppet-agent[15810]: (/Stage[main]/Role::Labs::Instance/Mount[/home]) Dependency Exec[block-for-home-export] has failures: true
Jan 27 19:18:44 fridolin puppet-agent[15810]: (/Stage[main]/Role::Labs::Instance/Mount[/home]) Skipping because of failed dependencies
Jan 27 19:18:47 fridolin puppet-agent[15810]: Finished catalog run in 373.43 seconds

Trying to mount it manually also fails:

jkroll@fridolin:~$ sudo mount -t nfs4 -o rw,noatime,vers=4.0,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=10.68.19.142,lookupcache=none,local_lock=none,addr=10.64.37.10 labstore.svc.eqiad.wmnet:/project/catgraph/home /home
mount.nfs4: mounting labstore.svc.eqiad.wmnet:/project/catgraph/home failed, reason given by server: No such file or directory

If you need any other information let me know.
Maybe I need to configure something? IIRC, on the older instances, /home got NFS-mounted automatically after creation.

Event Timeline

jkroll raised the priority of this task from to Needs Triage.
jkroll updated the task description. (Show Details)
jkroll added a project: Cloud-Services.
jkroll subscribed.
yuvipanda claimed this task.
yuvipanda subscribed.

This is because the nfs-exports service on labstore was dead for 3 days. I've started it now, and your next puppet run should give you /home.