Page MenuHomePhabricator

zuul_2.0.0-327-g3ebedde-wmf1precise1 fails importing daemon pidfile/pidlockfile
Closed, ResolvedPublic

Description

Weird packaging after bumping to python-daemon 2.0.4 shipped via the env

root@integration-zuul-server:~# dpkg -i zuul_2.0.0-327-g3ebedde-wmf1precise1_amd64.deb 
(Reading database ... 57388 files and directories currently installed.)
Preparing to replace zuul 2.0.0-304-g685ca22-wmf1precise1 (using zuul_2.0.0-327-g3ebedde-wmf1precise1_amd64.deb) ...
Unpacking replacement zuul ...
Setting up zuul (2.0.0-327-g3ebedde-wmf1precise1) ...
Traceback (most recent call last):
  File "/usr/bin/zuul-server", line 6, in <module>
    from zuul.cmd.server import main
  File "/usr/share/python/zuul/local/lib/python2.7/site-packages/zuul/cmd/server.py", line 23, in <module>
    pid_file_module = extras.try_imports(['daemon.pidlockfile', 'daemon.pidfile'])
  File "/usr/lib/python2.7/dist-packages/extras/__init__.py", line 92, in try_imports
    "Could not import any of: %s" % ', '.join(module_names))
ImportError: Could not import any of: daemon.pidlockfile, daemon.pidfile
Traceback (most recent call last):
  File "/usr/bin/zuul-merger", line 6, in <module>
    from zuul.cmd.merger import main
  File "/usr/share/python/zuul/local/lib/python2.7/site-packages/zuul/cmd/merger.py", line 23, in <module>
    pid_file_module = extras.try_imports(['daemon.pidlockfile', 'daemon.pidfile'])
  File "/usr/lib/python2.7/dist-packages/extras/__init__.py", line 92, in try_imports
    "Could not import any of: %s" % ', '.join(module_names))
ImportError: Could not import any of: daemon.pidlockfile, daemon.pidfile
Processing triggers for ureadahead ...
root@integration-zuul-server:~# pu^C
root@integration-zuul-server:~#

Event Timeline

hashar claimed this task.
hashar raised the priority of this task from to Needs Triage.
hashar updated the task description. (Show Details)
hashar subscribed.

Loading zuul-merger with verbose: /usr/share/python/zuul/bin/python -vv /usr/bin/zuul-merger :

import daemon # precompiled from /usr/share/python/zuul/local/lib/python2.7/site-packages/daemon/__init__.pyc
import __future__ # precompiled from /usr/lib/python2.7/__future__.pyc
import daemon.daemon # precompiled from /usr/share/python/zuul/local/lib/python2.7/site-packages/daemon/daemon.pyc

In daemon.pidfile:

from lockfile.pidlockfile import PIDLockFile

Which as a rather long list of attempts:

# trying /usr/share/python/zuul/bin/lockfile.so
# trying /usr/share/python/zuul/bin/lockfilemodule.so
# trying /usr/share/python/zuul/bin/lockfile.py
# trying /usr/share/python/zuul/bin/lockfile.pyc
# trying /usr/share/python/zuul/lib/python2.7/lockfile.so
# trying /usr/share/python/zuul/lib/python2.7/lockfilemodule.so
# trying /usr/share/python/zuul/lib/python2.7/lockfile.py
# trying /usr/share/python/zuul/lib/python2.7/lockfile.pyc
# trying /usr/share/python/zuul/lib/python2.7/lib-dynload/lockfile.so
# trying /usr/share/python/zuul/lib/python2.7/lib-dynload/lockfilemodule.so
# trying /usr/share/python/zuul/lib/python2.7/lib-dynload/lockfile.py
# trying /usr/share/python/zuul/lib/python2.7/lib-dynload/lockfile.pyc
# trying /usr/lib/python2.7/lockfile.so
# trying /usr/lib/python2.7/lockfilemodule.so
# trying /usr/lib/python2.7/lockfile.py
# trying /usr/lib/python2.7/lockfile.pyc
# trying /usr/lib/python2.7/plat-linux2/lockfile.so
# trying /usr/lib/python2.7/plat-linux2/lockfilemodule.so
# trying /usr/lib/python2.7/plat-linux2/lockfile.py
# trying /usr/lib/python2.7/plat-linux2/lockfile.pyc
# trying /usr/lib/python2.7/lib-tk/lockfile.so
# trying /usr/lib/python2.7/lib-tk/lockfilemodule.so
# trying /usr/lib/python2.7/lib-tk/lockfile.py
# trying /usr/lib/python2.7/lib-tk/lockfile.pyc
# trying /usr/share/python/zuul/local/lib/python2.7/site-packages/lockfile.so
# trying /usr/share/python/zuul/local/lib/python2.7/site-packages/lockfilemodule.so
# trying /usr/share/python/zuul/local/lib/python2.7/site-packages/lockfile.py
# trying /usr/share/python/zuul/local/lib/python2.7/site-packages/lockfile.pyc
# trying /usr/share/python/zuul/lib/python2.7/site-packages/lockfile.so
# trying /usr/share/python/zuul/lib/python2.7/site-packages/lockfilemodule.so
# trying /usr/share/python/zuul/lib/python2.7/site-packages/lockfile.py
# trying /usr/share/python/zuul/lib/python2.7/site-packages/lockfile.pyc
# trying /usr/local/lib/python2.7/site-packages/lockfile.so
# trying /usr/local/lib/python2.7/site-packages/lockfilemodule.so
# trying /usr/local/lib/python2.7/site-packages/lockfile.py
# trying /usr/local/lib/python2.7/site-packages/lockfile.pyc
# trying /usr/local/lib/python2.7/dist-packages/lockfile.so
# trying /usr/local/lib/python2.7/dist-packages/lockfilemodule.so
# trying /usr/local/lib/python2.7/dist-packages/lockfile.py
# trying /usr/local/lib/python2.7/dist-packages/lockfile.pyc
# trying /usr/lib/python2.7/dist-packages/lockfile.so
# trying /usr/lib/python2.7/dist-packages/lockfilemodule.so
# trying /usr/lib/python2.7/dist-packages/lockfile.py
# /usr/lib/python2.7/dist-packages/lockfile.pyc matches /usr/lib/python2.7/dist-packages/lockfile.py
import lockfile # precompiled from /usr/lib/python2.7/dist-packages/lockfile.pyc

It ends up loading the old version of lock file provided by python-lockfile deb package which fails:

$ /usr/share/python/zuul/bin/python
>>> import daemon.pidfile
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/share/python/zuul/local/lib/python2.7/site-packages/daemon/pidfile.py", line 18, in <module>
    from lockfile.pidlockfile import PIDLockFile
ImportError: No module named pidlockfile

Need to incorporate some more recent lockfile version. In Zuul requirements.txt we have lockfile >= 0.8 and that is fulfilled by the .deb package. Some weird issue cause the dependency of python-daemon to not be taken in account.

With the bump to python-daemon 2.x, upstream removed the dependency upon lock file>=0.8 ( https://review.openstack.org/#/c/148252/ ). Will attempt to drop it as well and see whether dh-virtualenv manage to fulfill python-daemon dependency.

Change 226129 had a related patch set uploaded (by Hashar):
lockfile is no more used

https://gerrit.wikimedia.org/r/226129

Change 226129 merged by Hashar:
lockfile is no more used

https://gerrit.wikimedia.org/r/226129

Issue fixed on labs by dropping lockfile and shipping python-daemon 2.x in the venv.