Page MenuHomePhabricator

Upgrade jenkins-debian-glue on Jessie slaves from 0.13.0 to latest (0.17.0)
Closed, ResolvedPublic

Description

Status: need packages from https://people.wikimedia.org/~hashar/debs/jenkins-debian-glue_0.17.0/ to be put on apt.wm.o


debian-glue jobs are running on Jessie which for now have jenkins-debian-glue v0.13.0.

To be able to build Zuul we need to enable network access and that in turns require v0.14.0 or following:

$ git describe b9c9cfdab47a3afa10d7d4a4b6377dc3bea3057f
v0.13.0-1-gb9c9cfd
$ git show b9c9cfdab47a3afa10d7d4a4b6377dc3bea3057f

commit b9c9cfdab47a3afa10d7d4a4b6377dc3bea3057f
Author: nextime <franco@unixmedia.it>
Date:   Thu Jun 4 11:09:08 2015 +0200

    Add pbuilder USENETWORK=yes option also to cowbuilder_run

diff --git a/scripts/build-and-provide-package b/scripts/build-and-provide-package
index 26158c3..710e2a9 100755
--- a/scripts/build-and-provide-package
+++ b/scripts/build-and-provide-package
@@ -492,6 +492,11 @@ EOF
   local pbuilderrc=$(mktemp)
   echo "# pbuilder config file generated by jenkins-debian-glue on $(date)" > "$pbuilderrc"
 
+  # allow pbuilder networking
+  if [ -n "${PBUILDER_USENETWORK:-}" ] ; then
+    echo "USENETWORK=yes" >> "$pbuilderrc"
+  fi
+
   # allow setting main pbuilder configuration file from outside, then append data
   # as needed without actually writing anything to user-provided $PBUILDER_CONFIG
   if [ -n "${PBUILDER_CONFIG:-}" ] ; then

So need the package to be rebuild and pushed to apt.wikimedia.org . We currently have:

$ apt-cache policy jenkins-debian-glue
jenkins-debian-glue:
  Installed: 0.13.0
  Candidate: 0.13.0
  Version table:
 *** 0.13.0 0
       1001 http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/main amd64 Packages
        100 /var/lib/dpkg/status
     0.11.0 0
       1001 http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/thirdparty amd64 Packages

Event Timeline

I have updated our copy in operations/debs/jenkins-debian-glue.git and pushed tags.

Rebuild using a cowbuilder env set up via our puppet class package_builder with the apt hook to inject apt.wm.o etc then:

DIST=jessie-wikimedia gbp buildpackage --git-verbose --git-debian-branch=master --git-upstream-branch=master

Result at: https://people.wikimedia.org/~hashar/debs/jenkins-debian-glue_0.17.0/

That version only has two binary package:

jenkins-debian-glue_0.17.0_all.deb
jenkins-debian-glue-buildenv_0.17.0_all.deb

We will want to cleanup apt.wm.org from previous packages:

jenkins-debian-glue-buildenv-git
jenkins-debian-glue-buildenv-lintian
jenkins-debian-glue-buildenv-piuparts
jenkins-debian-glue-buildenv-taptools

Change 300600 had a related patch set uploaded (by Hashar):
contint: jenkins-debian-glue 0.17.0

https://gerrit.wikimedia.org/r/300600

Change 300600 merged by Dzahn:
contint: jenkins-debian-glue 0.17.0

https://gerrit.wikimedia.org/r/300600

Mentioned in SAL [2016-07-22T20:26:46Z] <hashar> T141114 upgraded jenkins-debian-glue from v0.13.0 to v0.17.0 on integration-slave-jessie-1001 and integration-slave-jessie-1002

I suppose we should upload the debs at the repo at some point ?

I got it installed manually on the permanent Jessie slaves and indeed that is now pending upload to apt.wikimedia.org see T141114#2488638

Need SRE to publish jenkins-debian-glue packages to apt.wikimedia.org T141114#2488638

hashar triaged this task as Medium priority.Aug 24 2016, 11:13 AM
hashar removed a project: Patch-For-Review.

Mentioned in SAL [2016-08-27T20:24:05Z] <hashar> Manually installing jenkins-debian-glue 0.17.0 on integration-slave-jessie-1004 and integration-slave-jessie-1005 ( T142891 ) . That is to support PBUILDER_USENETWORK T141114

Added to Puppet SWAT of Tuesday, August 30

Too many patches for today puppet swat. @fgiunchedi and I will sync tomorrow.

fgiunchedi claimed this task.

this is completed, package uploaded

# reprepro list jessie-wikimedia jenkins-debian-glue
jessie-wikimedia|main|amd64: jenkins-debian-glue 0.17.0
jessie-wikimedia|main|i386: jenkins-debian-glue 0.17.0
jessie-wikimedia|main|source: jenkins-debian-glue 0.17.0
jessie-wikimedia|thirdparty|amd64: jenkins-debian-glue 0.11.0
jessie-wikimedia|thirdparty|i386: jenkins-debian-glue 0.11.0

Confirmed. Since 0.17.0 got installed to main can you drop the old version from thirdparty please? That is to avoid potential confusion later on:

jessie-wikimediathirdpartyamd64: jenkins-debian-glue 0.11.0
jessie-wikimediathirdpartyi386: jenkins-debian-glue 0.11.0

Thx!

Sorry I have forgot about a bunch of other ones:

jenkins-debian-glue-buildenv-slave0.13.0http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/main amd64 Packages
jenkins-debian-glue-buildenv-slave0.11.0http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/thirdparty amd64 Packages
jenkins-debian-glue-buildenv0.11.0http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/thirdparty amd64 Packages
jenkins-debian-glue-buildenv-taptools0.13.0http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/main amd64 Packages
jenkins-debian-glue-buildenv-taptools0.11.0http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/thirdparty amd64 Packages
jenkins-debian-glue-buildenv-svn0.13.0http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/main amd64 Packages
jenkins-debian-glue-buildenv-svn0.11.0http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/thirdparty amd64 Packages
jenkins-debian-glue-buildenv-git0.13.0http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/main amd64 Packages
jenkins-debian-glue-buildenv-git0.11.0http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/thirdparty amd64 Packages
jenkins-debian-glue-buildenv-lintian0.13.0http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/main amd64 Packages
jenkins-debian-glue-buildenv-lintian0.11.0http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/thirdparty amd64 Packages
jenkins-debian-glue-buildenv-piuparts0.13.0http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/main amd64 Packages
jenkins-debian-glue-buildenv-piuparts0.11.0http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/thirdparty amd64 Packages

@hashar Also it seems the packages it depends on do not exist in debian package manager causing installs to fail.

@Paladox what do you mean? Do you have some kind of trace?

On Jessie apt-get update && apt-get install jenkins-debian-glue jenkins-debian-glue-buildenv should work. It seems to work properly on the CI Jessie slaves.

@Paladox what do you mean? Do you have some kind of trace?

On Jessie apt-get update && apt-get install jenkins-debian-glue jenkins-debian-glue-buildenv should work. It seems to work properly on the CI Jessie slaves.

@hashar When I applied the deb by doing dpkg -I <deb_name> it failed to install because of some debs wernt available on debian package manager, I had to manually find it and install the missing package then I also had to install the other package do to the failure.

It was two weeks ago I think.

@Paladox dpkg -i does not resolve dependencies so you then have to manually fix them.

Now that the package has been pushed on apt.wikimedia.org you just have to:

apt-get install jenkins-debian-glue  jenkins-debian-glue-buildenv

@fgiunchedi it is all set now:

$ sudo apt-get update; apt-cache madison 'jenkins-debian-glue*'
...
jenkins-debian-glue-buildenv |     0.17.0 | http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/main amd64 Packages
jenkins-debian-glue |     0.17.0 | http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/main amd64 Packages
$ 

Thank you very much.

And eventually I found out today that our 0.17.0 package include a few more patches from upstream:

* 09a78f2 - (gerrit/master, gerrit/HEAD) generate-git-snapshot: support DCH_CHANGELOG_FILE to change path for git-dch (1 year, 9 months ago) <Sebastian Damm>
* c1e13b9 - Adding SKIP_MISSING_BINARY_REMOVAL to prevent removal of binary packages not belonging to the currently built source package. (1 year, 9 months ago) <Antoine Delvaux>
* f9b1fc8 - puppet: update list of Jenkis plugins to satisfy dependencies (1 year, 10 months ago) <Michael Prokop>

I screwed up when generating the upstream tarball, using the master branch instead of v0.17.0 tag.

It is not much of an issue though. Will get the latest tagged version packaged and uploaded.