Page MenuHomePhabricator

Nodepool does not delete nodes in a timely manner (ZeroMQ dead)
Closed, ResolvedPublic

Description

Whenever a job is completed, the Gearman plugin properly put the node offline.But Nodepool doesn't notice it.

The reason is that the Nodepool / Jenkins ZeroMQ bus is dead. Jenkins no more listen on port 8888 and Nodepool kindly ignore the fact it can't receive any ZMQ events.

Event Timeline

hashar created this task.Dec 7 2015, 4:39 PM
hashar raised the priority of this task from to Needs Triage.
hashar updated the task description. (Show Details)
hashar added a subscriber: hashar.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptDec 7 2015, 4:39 PM
hashar added a comment.Dec 7 2015, 4:45 PM

That is probably because when restarting Jenkins there were duplicate process. Thus on the second instance ZMQ would have failed because TCP port 8888 was already bound.

hashar closed this task as Resolved.Dec 7 2015, 5:00 PM
hashar claimed this task.

I have restarted Jenkins to get the ZeroMQ publisher started again.