We currently run Jenkins 2.319.3, we should upgrade to the next LTS 2.332.2
http://www.jenkins.io/changelog-stable/
Looks like the Debian package switch from hold System V init to systemd: https://issues.jenkins.io/browse/JENKINS-41218 Maybe we can consume their script instead of our own.
The Debian package can be retrieved from upstream and pushed to our apt.wikimedia.org by following the instructions at https://wikitech.wikimedia.org/wiki/Jenkins#Get_the_package
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| jenkins: update path to war file | operations/puppet | production | +2 -2 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Restricted Task | |||||
| Resolved | hashar | T307339 Upgrade Jenkins to the next LTS, 2.332.2 | |||
| Resolved | hashar | T308637 Use Jenkins upstream systemd unit instead of our own |
Change 792974 had a related patch set uploaded (by Hashar; author: Hashar):
[operations/puppet@production] jenkins: update path to war file
The package ship a /etc/default/jenkins which our Puppet erase and also comes with a systemd unit which we might consider using instead of our Puppet one.
I worked around one of the issue with:
sudo ln -s /usr/share/java/jenkins.war /usr/share/jenkins/jenkins.war
The agents could not connect:
[05/18/22 08:19:31] [SSH] Starting sftp client. [05/18/22 08:19:32] [SSH] Copying latest remoting.jar... java.io.IOException: Could not copy remoting.jar into '/srv/jenkins-workspace' on agent at hudson.plugins.sshslaves.SSHLauncher.copyAgentJar(SSHLauncher.java:717) at hudson.plugins.sshslaves.SSHLauncher.access$300(SSHLauncher.java:113) at hudson.plugins.sshslaves.SSHLauncher$1.call(SSHLauncher.java:457) at hudson.plugins.sshslaves.SSHLauncher$1.call(SSHLauncher.java:423) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.IllegalArgumentException: invalid len argument at com.trilead.ssh2.SFTPv3Client.read(SFTPv3Client.java:1232) at com.trilead.ssh2.jenkins.SFTPClient$SFTPInputStream.read(SFTPClient.java:172) at com.google.common.io.ByteStreams.toByteArrayInternal(ByteStreams.java:184) at com.google.common.io.ByteStreams.toByteArray(ByteStreams.java:224) at hudson.plugins.sshslaves.SSHLauncher.readInputStreamIntoByteArrayAndClose(SSHLauncher.java:775) at hudson.plugins.sshslaves.SSHLauncher.copyAgentJar(SSHLauncher.java:686) ... 7 more [05/18/22 08:19:32] Launch failed - cleaning up connection
I have manually deleted them from integration-cumin with:
sudo cumin --force '*' 'rm /srv/jenkins/workspace/remoting.jar'
The issue is https://issues.jenkins.io/browse/JENKINS-67165 . Apparently requires ssh-slaves-plugin to be updated for https://github.com/jenkinsci/ssh-slaves-plugin/pull/228 which is in 1.32.0 but we run 1.31.5 on CI Jenkins :-\ The release Jenkins was not affected and it uses 1.32.0
Mentioned in SAL (#wikimedia-releng) [2022-05-18T08:29:02Z] <hashar> Updating SSH Build agent from 1.31.5 to 1.32.0 on CI Jenkins to prevent an issue when uploading remoting.jar # T307339#7937268
Change 792974 merged by Muehlenhoff:
[operations/puppet@production] jenkins: update path to war file