Page MenuHomePhabricator
Create Task
Maniphest T106399

zuul_2.0.0-327-g3ebedde-wmf1precise1 fails importing daemon pidfile/pidlockfile
Closed, ResolvedPublic
Actions

Description

Weird packaging after bumping to python-daemon 2.0.4 shipped via the env

root@integration-zuul-server:~# dpkg -i zuul_2.0.0-327-g3ebedde-wmf1precise1_amd64.deb 
(Reading database ... 57388 files and directories currently installed.)
Preparing to replace zuul 2.0.0-304-g685ca22-wmf1precise1 (using zuul_2.0.0-327-g3ebedde-wmf1precise1_amd64.deb) ...
Unpacking replacement zuul ...
Setting up zuul (2.0.0-327-g3ebedde-wmf1precise1) ...
Traceback (most recent call last):
  File "/usr/bin/zuul-server", line 6, in <module>
    from zuul.cmd.server import main
  File "/usr/share/python/zuul/local/lib/python2.7/site-packages/zuul/cmd/server.py", line 23, in <module>
    pid_file_module = extras.try_imports(['daemon.pidlockfile', 'daemon.pidfile'])
  File "/usr/lib/python2.7/dist-packages/extras/__init__.py", line 92, in try_imports
    "Could not import any of: %s" % ', '.join(module_names))
ImportError: Could not import any of: daemon.pidlockfile, daemon.pidfile
Traceback (most recent call last):
  File "/usr/bin/zuul-merger", line 6, in <module>
    from zuul.cmd.merger import main
  File "/usr/share/python/zuul/local/lib/python2.7/site-packages/zuul/cmd/merger.py", line 23, in <module>
    pid_file_module = extras.try_imports(['daemon.pidlockfile', 'daemon.pidfile'])
  File "/usr/lib/python2.7/dist-packages/extras/__init__.py", line 92, in try_imports
    "Could not import any of: %s" % ', '.join(module_names))
ImportError: Could not import any of: daemon.pidlockfile, daemon.pidfile
Processing triggers for ureadahead ...
root@integration-zuul-server:~# pu^C
root@integration-zuul-server:~#

Details

SubjectRepoBranchLines +/-
lockfile is no more usedintegration/zuuldebian/precise-wikimedia+33 -4
Customize query in gerrit

Related Objects

Event Timeline

hashar created this task.Jul 21 2015, 6:03 PM
hashar claimed this task.
hashar raised the priority of this task from to Needs Triage.
hashar updated the task description. (Show Details)
hashar added projects: Continuous-Integration-Infrastructure, Zuul.
hashar subscribed.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 21 2015, 6:03 PM
hashar added a comment.Jul 21 2015, 7:09 PM

Loading zuul-merger with verbose: /usr/share/python/zuul/bin/python -vv /usr/bin/zuul-merger :

import daemon # precompiled from /usr/share/python/zuul/local/lib/python2.7/site-packages/daemon/__init__.pyc
import __future__ # precompiled from /usr/lib/python2.7/__future__.pyc
import daemon.daemon # precompiled from /usr/share/python/zuul/local/lib/python2.7/site-packages/daemon/daemon.pyc

In daemon.pidfile:

from lockfile.pidlockfile import PIDLockFile

Which as a rather long list of attempts:

# trying /usr/share/python/zuul/bin/lockfile.so
# trying /usr/share/python/zuul/bin/lockfilemodule.so
# trying /usr/share/python/zuul/bin/lockfile.py
# trying /usr/share/python/zuul/bin/lockfile.pyc
# trying /usr/share/python/zuul/lib/python2.7/lockfile.so
# trying /usr/share/python/zuul/lib/python2.7/lockfilemodule.so
# trying /usr/share/python/zuul/lib/python2.7/lockfile.py
# trying /usr/share/python/zuul/lib/python2.7/lockfile.pyc
# trying /usr/share/python/zuul/lib/python2.7/lib-dynload/lockfile.so
# trying /usr/share/python/zuul/lib/python2.7/lib-dynload/lockfilemodule.so
# trying /usr/share/python/zuul/lib/python2.7/lib-dynload/lockfile.py
# trying /usr/share/python/zuul/lib/python2.7/lib-dynload/lockfile.pyc
# trying /usr/lib/python2.7/lockfile.so
# trying /usr/lib/python2.7/lockfilemodule.so
# trying /usr/lib/python2.7/lockfile.py
# trying /usr/lib/python2.7/lockfile.pyc
# trying /usr/lib/python2.7/plat-linux2/lockfile.so
# trying /usr/lib/python2.7/plat-linux2/lockfilemodule.so
# trying /usr/lib/python2.7/plat-linux2/lockfile.py
# trying /usr/lib/python2.7/plat-linux2/lockfile.pyc
# trying /usr/lib/python2.7/lib-tk/lockfile.so
# trying /usr/lib/python2.7/lib-tk/lockfilemodule.so
# trying /usr/lib/python2.7/lib-tk/lockfile.py
# trying /usr/lib/python2.7/lib-tk/lockfile.pyc
# trying /usr/share/python/zuul/local/lib/python2.7/site-packages/lockfile.so
# trying /usr/share/python/zuul/local/lib/python2.7/site-packages/lockfilemodule.so
# trying /usr/share/python/zuul/local/lib/python2.7/site-packages/lockfile.py
# trying /usr/share/python/zuul/local/lib/python2.7/site-packages/lockfile.pyc
# trying /usr/share/python/zuul/lib/python2.7/site-packages/lockfile.so
# trying /usr/share/python/zuul/lib/python2.7/site-packages/lockfilemodule.so
# trying /usr/share/python/zuul/lib/python2.7/site-packages/lockfile.py
# trying /usr/share/python/zuul/lib/python2.7/site-packages/lockfile.pyc
# trying /usr/local/lib/python2.7/site-packages/lockfile.so
# trying /usr/local/lib/python2.7/site-packages/lockfilemodule.so
# trying /usr/local/lib/python2.7/site-packages/lockfile.py
# trying /usr/local/lib/python2.7/site-packages/lockfile.pyc
# trying /usr/local/lib/python2.7/dist-packages/lockfile.so
# trying /usr/local/lib/python2.7/dist-packages/lockfilemodule.so
# trying /usr/local/lib/python2.7/dist-packages/lockfile.py
# trying /usr/local/lib/python2.7/dist-packages/lockfile.pyc
# trying /usr/lib/python2.7/dist-packages/lockfile.so
# trying /usr/lib/python2.7/dist-packages/lockfilemodule.so
# trying /usr/lib/python2.7/dist-packages/lockfile.py
# /usr/lib/python2.7/dist-packages/lockfile.pyc matches /usr/lib/python2.7/dist-packages/lockfile.py
import lockfile # precompiled from /usr/lib/python2.7/dist-packages/lockfile.pyc

It ends up loading the old version of lock file provided by python-lockfile deb package which fails:

$ /usr/share/python/zuul/bin/python
>>> import daemon.pidfile
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/share/python/zuul/local/lib/python2.7/site-packages/daemon/pidfile.py", line 18, in <module>
    from lockfile.pidlockfile import PIDLockFile
ImportError: No module named pidlockfile

Need to incorporate some more recent lockfile version. In Zuul requirements.txt we have lockfile >= 0.8 and that is fulfilled by the .deb package. Some weird issue cause the dependency of python-daemon to not be taken in account.

With the bump to python-daemon 2.x, upstream removed the dependency upon lock file>=0.8 ( https://review.openstack.org/#/c/148252/ ). Will attempt to drop it as well and see whether dh-virtualenv manage to fulfill python-daemon dependency.

gerritbot subscribed.Jul 21 2015, 7:34 PM

Change 226129 had a related patch set uploaded (by Hashar):
lockfile is no more used

https://gerrit.wikimedia.org/r/226129

gerritbot added a project: Patch-For-Review.Jul 21 2015, 7:34 PM
gerritbot added a comment.Jul 21 2015, 7:44 PM

Change 226129 merged by Hashar:
lockfile is no more used

https://gerrit.wikimedia.org/r/226129

hashar mentioned this in rCIZU4be740688b14: lockfile is no more used.Jul 21 2015, 7:44 PM
hashar closed this task as Resolved.Jul 21 2015, 7:45 PM

Issue fixed on labs by dropping lockfile and shipping python-daemon 2.x in the venv.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits