Page MenuHomePhabricator

Upload new Zuul packages on apt.wikimedia.org for Precise / Trusty / Jessie
Closed, ResolvedPublic

Description

TL;DR would need new Zuul .deb to be pushed to apt.wikimedia.org for Precise/Trusty/Jessie

Last week I finally went to upgrade Zuul to the latest version. It is a tedious process to have the same code base to support all three distributions, most notably the python packages are severely outdated. We went with dh_virtualenv which let us easily embed dependencies in the Zuul packages.

I already upgraded:

  • Zuul scheduler and merger instances on gallium (Precise).
  • All the CI slaves (Precise/Trusty)

Left to do is:

  • installing Zuul on the Jessie slaves (we lacked a package for Jessie)
  • Provision Zuul client on the Nodepool instances which are Jessie
  • Get Zuul installed on scandium, the new merger (Jessie) T95046

The breakdown is:

Note how Ubuntu uses main whereas with Jessie we now uses thirdparty.

All three packages are based on the same tarball zuul_2.1.0-60-g1cc37f7.orig.tar.gz which is upstream commit 1cc37f7b469a892cdbd16db6aa1d500a1200c417

Should be no impact on production since I already upgraded the packages.

Event Timeline

hashar created this task.Nov 10 2015, 10:32 PM
hashar claimed this task.
hashar raised the priority of this task from to Normal.
hashar updated the task description. (Show Details)
hashar added subscribers: Andrew, chasemp, Aklapper, fgiunchedi.
hashar updated the task description. (Show Details)Nov 13 2015, 1:30 PM
hashar set Security to None.

I had to bump the patch 0005-Cloner-Implement-cache-no-hardlinks-argument.patch which was outdated. That is for T97106.

I have updated the task detail and the new versions are zuul_2.1.0-60-g1cc37f7-wmf2<DISRO>1

I am upgrading it on gallium and the CI slaves.

I'd be a bit happier if the source tree that these packages are built from is checked into gerrit, with clearly labeled branches corresponding to the built packages. That's pretty easy, I think.

These are packages that will only be installed on labs instances, right?

The packaging work is held in our Gerrit repo integration/zuul.git with the following branches:

upstream1cc37f7b469a892cdbd16db6aa1d500a1200c417 == 2.1.0-60-g1cc37f7
debian/precise-wikimediaHold /debian and sources patches maintained by gbp pq
debian/trusty-wikimediaFork for Trusty, kept in sync with precise-wikimedia by merging
debian/jessie-wikimediaFork for Jessie, kept in sync with precise-wikimedia by merging

The source patches are maintained using gbp pq in the branch patch-queue/debian/precise-wikimedia.

Whenever I bump upstream, I:

  • bump head of upstream in Gerrit
  • merge upstream into debian/precise-wikimedia
  • use gbp pq rebase to rebase patch-queue/debian/precise-wikimedia. When patches got merged they disappear
  • gbp pq export to resynchronize the patches in debian/precise-wikimedia
  • create a new package

For the Trusty and Jessie distribution, I then merge debian/precise-wikimedia into them and rebuild them.

The packages are mean to be installed on all permanent slaves and the Nodepool slaves (Jessie).

The packages are used on production, we haven't had a good way to match upstream requirements and I went using dh_virtualenv to ship them in the .deb. The usage in production is as follow:

hostdistrorole(s)
galliumPreciseZuul scheduler and merger
scandiumJessieZuul merger

ok, good enough for me :)

hashar closed this task as Resolved.Nov 17 2015, 4:02 PM

Andrew uploaded them all :-} Thank you!