Page MenuHomePhabricator

Zuul votes Verified-1 on all changes because it is unable to merge patchset on tip of branch (zuul-merger issue)
Closed, ResolvedPublic

Description

CI is broken, the zuul-merger can't not merge anything:

2015-10-12 08:41:58,952 ERROR zuul.Merger: Unable to reset repo <zuul.merger.merger.Repo object at 0x7f7c741a2690>
Traceback (most recent call last):
  File "/usr/share/python/zuul/local/lib/python2.7/site-packages/zuul/merger/merger.py", line 285, in _mergeItem
    repo.reset()
  File "/usr/share/python/zuul/local/lib/python2.7/site-packages/zuul/merger/merger.py", line 87, in reset
    self.update()
  File "/usr/share/python/zuul/local/lib/python2.7/site-packages/zuul/merger/merger.py", line 179, in update
    origin = repo.remotes.origin
  File "/usr/share/python/zuul/local/lib/python2.7/site-packages/git/util.py", line 669, in __getattr__
    return list.__getattribute__(self, attr)
AttributeError: 'IterableList' object has no attribute 'origin'

Event Timeline

hashar claimed this task.
hashar raised the priority of this task from to Unbreak Now!.
hashar updated the task description. (Show Details)
hashar added a subscriber: hashar.

Using strace:

[pid 25328] open("/srv/ssd/zuul/git/integration/dashboard/.git/config", O_RDONLY) = -1 EMFILE (Too many open files)
lsof -p 25304|wc -l
2061

zuul-merger leaks file descriptors :-( It keeps .idx / .pack files open for some reason :-/

I have restarted the zuul-merger process on gallium and it is back.