Page MenuHomePhabricator

Library restart detection is very slow in Kubernetes workers
Open, LowPublic

Description

debdeploy uses lsof to detect file references to deletes library file descriptors (which indicate a library that has been updated, but still is kept in memory by the dynamic linker since a process is still using it). This takes a few deci seconds on all servers, but on the Kubernetes workers is notably slow:

jmm@wikikube-worker1266:~$ time sudo lsof +c 15 -lnXd DEL
real    0m34.487s
user    0m10.342s
sys     0m24.026s

Maybe this is caused by the overlayfs mounts, needs some further investigation

Event Timeline

I've already tried to make lsof exclude bunch of mountpoints with:
lsof -nXd DEL $(findmnt -t tmpfs,nsfs,overlay,proc,^Csfs,cgroup2,devtmpfs,devpts,securityfs,pstore,bpf,hugetlbfs,mqueue,debugfs,tracefs,fusectl,configfs,ramfs, -o TARGET -n --list | sed 's/^/-e /')

But that did not change the time it takes to run at all.