Page MenuHomePhabricator

wmf-auto-restart can get wedged on nfs4 mounts even when the filesystem is excluded
Closed, InvalidPublic

Description

I thought I had this fixed in T336845: puppet: profile::auto_restarts::service: have a way to don't deploy the systemd timers though apparently not, e.g.

root     2995022  0.0  0.0  18072 13576 ?        Ss   03:23   0:00 /usr/bin/python3 /usr/local/sbin/wmf-auto-restart -s ssh
root     2995023  0.0  0.0   5828  3240 ?        S    03:23   0:00  \_ /usr/bin/lsof +c 15 -nXd DEL -e /mnt/nfs/dumps-clouddumps1002.wikimedia.org -e /mnt/nfs/dumps-clouddumps1001.wikimedia.org -e /mnt/nfs/labstore-secondary-tools-project -e /mnt/nfs/secondary-scratch -e /mnt/nfs/labstore-secondary-tools-home
root     2995024  0.0  0.0   5564   292 ?        D    03:23   0:00      \_ /usr/bin/lsof +c 15 -nXd DEL -e /mnt/nfs/dumps-clouddumps1002.wikimedia.org -e /mnt/nfs/dumps-clouddumps1001.wikimedia.org -e /mnt/nfs/labstore-secondary-tools-project -e /mnt/nfs/secondary-scratch -e /mnt/nfs/labstore-secondary-tools-home
root@tools-k8s-worker-nfs-53:~# cat /proc/2995024/stack
[<0>] folio_wait_bit_common+0x13d/0x350
[<0>] folio_wait_writeback+0x28/0x80
[<0>] __filemap_fdatawait_range+0x90/0x120
[<0>] filemap_write_and_wait_range+0x5f/0x80
[<0>] nfs_getattr+0x422/0x470 [nfs]
[<0>] vfs_statx+0xc2/0x180
[<0>] vfs_fstatat+0x51/0x70
[<0>] __do_sys_newfstatat+0x3f/0x80
[<0>] do_syscall_64+0x55/0x80
[<0>] entry_SYSCALL_64_after_hwframe+0x6e/0xd8