Page MenuHomePhabricator

have Jenkins build Debian packages for us
Closed, ResolvedPublic


mark / ops could use some automatic job to build Debian packages from git using git-buildpackage.

A possible shell script we could use :

Version: unspecified
Severity: enhancement



Event Timeline

bzimport raised the priority of this task from to Low.Nov 22 2014, 12:28 AM
bzimport set Reference to bz36443.

Can be smashed during the Berlin Hackaton 2012.

Not going to happen till I figure out how build a deb package out of the operations/debs/* repos. If anyone has a step by step manual or a shell script, you are welcome to cast your voice on this report.

Also marking this has an enhancement request.

I can probably help with this in the next couple of weeks as I have some free time.

AzaToth mentioned 'jenkins-debian-glue' which is a wrapper to easily integration debian building in Jenkins.

The step by step guide is at

A useful FAQ entry:

I have created the integration-debian-builder.pmtpa.wmflabs instance to be used as a build box with jenkins-deploy as a user and an appropriate sudo policy. The workspace is set to /mnt/build which is vdb and has lot of free space.

The demo job is working properly and managed to generate a debian package for pybal.

Now I need to:

  • probably use a different username in labs than jenkins-deploy
  • puppetize the installation of a Jenkins slave in labs
  • puppetize the basic file layout that is going to be used in labs (/mnt/build and /mnt/repository)
  • write YAML definitions in Jenkins Job Builder, making sure to reproduce the various hack I made in the jobs such as overriding the workspace to point to /mnt/build instead of the tiny /home/jenkins-deploy
  • generate jobs and try them out
  • write Zuul triggers for the repositories. Probably needs to be non voting

Finally, publish the build packages in a reprepro repository . Either on gallium or somewhere else.

I have created RT #5394 to get jenkins-debian-glue package uploaded on apt.wm.o

Change 71245 had a related patch set uploaded by AzaToth:
Add debian-glue macros

Change 71245 abandoned by AzaToth:
Add debian-glue macros

Change 71245 restored by AzaToth:
Add debian-glue macros

Change 84931 had a related patch set uploaded by Hashar:
trigger jenkins-debian-glue jobs for pybal

Change 84931 merged by jenkins-bot:
trigger jenkins-debian-glue jobs for pybal

My test change is which is against operations/debs/pybal.git
The related jobs are:
Only the -source job is triggered by Zuul for now, thus only the source result is submitted back in Gerrit. We will want Zuul to triggers source, binaries and piuparts one after the other and tweak the job so they do not trigger each other. That is going to cause some dependencies trouble though, we want to make sure each build uses the proper artifacts and not the one from a job that have been launched in between.
The Jenkins slave is integration-debian-builder.pmtpa.wmflabs and it has a bunch of .deb in its repository:
$ ls -1 /mnt/repository/pool/main/p/pybal/
Congratulations Carl, we are almost there :-] Thank you very much!

Summary from IRC:

If I send 3 changes A,B,C , by the time the 'binaries' job for A is completed, the 'source' job last artifacts would be the one of change C. Hence, when 'binaries' job for B runs, it will end up fetching the artifacts of C.

all jobs are sharing the same build parameter ZUUL_COMMIT that is a merge commit of the patchset against the tip of the branch. Potentially the copy artifact plugin could look up in the other job for the last successful build that has the same ZUUL_COMMIT value.

Change 71245 merged by jenkins-bot:
debian-glue macros and jobs

The infrastructure has been setup. The Jenkins job builder template '{name}-debian-glue' can be used to run git buildpackage / cowbuilder.