Page MenuHomePhabricator

Move debian-glue jobs to Docker using Stretch as a base image
Closed, DeclinedPublic

Description

We run the debian packaging jobs using debian-glue on integration-slave-jessie* instances. Would be great to move them to standalone slaves. Potentially upgrade them from Jessie to Stretch while at it.

On Jessie we have 0.17.0

$ apt-cache policy jenkins-debian-glue
jenkins-debian-glue:
  Installed: 0.17.0
  Candidate: 0.17.0
  Version table:
 *** 0.17.0 0
       1001 http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/main amd64 Packages
        100 /var/lib/dpkg/status

Stretch comes with 0.18.2

Event Timeline

hashar renamed this task from Move debian-glue jobs to dedicated slaves to Move debian-glue jobs to Docker using Stretch as a base image.Jan 1 2019, 3:52 PM
hashar changed the task status from Open to Stalled.Jan 9 2019, 3:01 PM

That starts well: Debian #901723 cowbuilder not working properly under docker:

I: Installing apt-lines
I: Refreshing the base.tgz
I: upgrading packages
I: mounting /proc filesystem
mount: /var/cache/pbuilder/base.cow/proc: permission denied.
W: Aborting with an error
E: pbuilder create failed
I: forking: rm -rf /var/cache/pbuilder/base.cow
The command '/bin/sh -c git-pbuilder create' returned a non-zero code: 32

Running as root in the container ..

We would also need to get material from puppet modules/package_builder . It is not too complicated to achieve:

FROM {{ "ci-jessie" | image_tag }}

# XXX
# Make sure we get our forked version 0.18.4-wmf3 from jessie-wikimedia/main
RUN {{ "jenkins-debian-glue jenkins-debian-glue-buildenv" | apt_install }}

RUN {{ "puppet" | apt_install }}
RUN git clone --depth 1 https://gerrit.wikimedia.org/r/p/operations/puppet.git /tmp/puppet

RUN cd /tmp/puppet/modules/package_builder \
    && puppet apply --basemodulepath=.. --test --verbose --logdest console --execute 'include package_builder' \
    && apt-get -y remove --purge puppet \
    && apt-get -y autoremove --purge

But that creates cow images for each of the supported distributions, resulting in a fairly large container. It would probably be smarter to have them on the hosts and just volume mount (read only) inside the container.
Marking this task stalled there are other tasks in Continuous-Integration-Infrastructure (Slipway) we can do before deciding the future of debian-glue jobs.

greg triaged this task as Medium priority.Jan 16 2019, 4:38 PM

we cant quite run cowbuider inside a Docker container :/