Jenkins is still blocking on /etc/init.d/jenkins stop. I took a jstack for it:
Attaching to process ID 4217, please wait... Debugger attached successfully. Server compiler detected. JVM version is 24.75-b04 Deadlock Detection: No deadlocks found. Thread 31383: (state = BLOCKED) - java.lang.Shutdown.exit(int) @bci=96, line=212 (Interpreted frame) - java.lang.Terminator$1.handle(sun.misc.Signal) @bci=8, line=52 (Interpreted frame) - sun.misc.Signal$1.run() @bci=8, line=212 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 30581: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=186 (Interpreted frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=834 (Interpreted frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(int) @bci=72, line=994 (Interpreted frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(int) @bci=24, line=1303 (Interpreted frame) - java.util.concurrent.CountDownLatch.await() @bci=5, line=236 (Interpreted frame) - org.pircbotx.Channel.getMode() @bci=76, line=127 (Interpreted frame) - org.pircbotx.Channel.getModeArgument(char) @bci=1, line=182 (Interpreted frame) - org.pircbotx.Channel.getChannelKey() @bci=3, line=239 (Interpreted frame) - org.pircbotx.PircBotX.shutdown(boolean) @bci=112, line=2872 (Interpreted frame) - org.pircbotx.PircBotX$4.run() @bci=38, line=2817 (Interpreted frame) Thread 30574: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - java.lang.Thread.join(long) @bci=38, line=1281 (Interpreted frame) - java.lang.Thread.join() @bci=2, line=1355 (Interpreted frame) - java.lang.ApplicationShutdownHooks.runHooks() @bci=87, line=106 (Interpreted frame) - java.lang.ApplicationShutdownHooks$1.run() @bci=0, line=46 (Interpreted frame) - java.lang.Shutdown.runHooks() @bci=39, line=123 (Interpreted frame) - java.lang.Shutdown.sequence() @bci=26, line=167 (Interpreted frame) - java.lang.Shutdown.exit(int) @bci=96, line=212 (Interpreted frame) - java.lang.Terminator$1.handle(sun.misc.Signal) @bci=8, line=52 (Interpreted frame) - sun.misc.Signal$1.run() @bci=8, line=212 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 19094: (state = IN_NATIVE) - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 (Compiled frame; information may be imprecise) - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Compiled frame) - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=79 (Compiled frame) - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=87 (Compiled frame) - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=98 (Compiled frame) - sun.nio.ch.SelectorImpl.select() @bci=2, line=102 (Compiled frame) - hudson.plugins.gearman.MyGearmanWorkerImpl.work() @bci=606, line=340 (Compiled frame) - hudson.plugins.gearman.AbstractWorkerThread.run() @bci=103, line=166 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 5377: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=186 (Interpreted frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() @bci=42, line=2043 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=442 (Compiled frame) - org.jenkinsci.plugins.ZMQEventPublisher.ZMQRunnable.run() @bci=4, line=95 (Compiled frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 5373: (state = IN_NATIVE) - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 (Interpreted frame) - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Interpreted frame) - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=79 (Interpreted frame) - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=87 (Interpreted frame) - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=98 (Interpreted frame) - zmq.Poller.run() @bci=153, line=202 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 5372: (state = IN_NATIVE) - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 (Interpreted frame) - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Interpreted frame) - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=79 (Interpreted frame) - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=87 (Interpreted frame) - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=98 (Interpreted frame) - zmq.Poller.run() @bci=153, line=202 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 4362: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=186 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() @bci=42, line=2043 (Compiled frame) - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() @bci=24, line=1079 (Interpreted frame) - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() @bci=1, line=807 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=156, line=1068 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1130 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=615 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 4361: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=186 (Interpreted frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=834 (Interpreted frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(int) @bci=72, line=994 (Interpreted frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(int) @bci=24, line=1303 (Interpreted frame) - java.util.concurrent.Semaphore.acquire() @bci=5, line=317 (Interpreted frame) - hudson.plugins.im.IMConnectionProvider$ConnectorRunnable.run() @bci=4, line=160 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 4227: (state = BLOCKED) Thread 4334: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - java.lang.ref.ReferenceQueue.remove(long) @bci=44, line=135 (Compiled frame) - java.lang.ref.ReferenceQueue.remove() @bci=2, line=151 (Compiled frame) - sun.java2d.Disposer.run() @bci=3, line=145 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 4239: (state = BLOCKED) Thread 4238: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - java.lang.ref.ReferenceQueue.remove(long) @bci=44, line=135 (Compiled frame) - java.lang.ref.ReferenceQueue.remove() @bci=2, line=151 (Compiled frame) - java.lang.ref.Finalizer$FinalizerThread.run() @bci=36, line=209 (Compiled frame) Thread 4237: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - java.lang.Object.wait() @bci=2, line=503 (Compiled frame) - java.lang.ref.Reference$ReferenceHandler.run() @bci=46, line=133 (Compiled frame)
Seems like the IRC notification plugin deadlocks the shutdown waiting for org.pircbotx.Channel.getMode. Might be similar to T96183: Jenkins: Builds (for beta cluster and browser tests) are stuck forever if IRC notification failed.