Page MenuHomePhabricator

cowbuilder cron update fails for buster
Closed, ResolvedPublic

Description

integration-slave-jessie CI instances use the Puppet package_builder which generates some base cow images.

I had a build targeting buster that fails with:

E: Failed to fetch http://mirrors.wikimedia.org/debian/pool/main/s/systemd/libsystemd-dev_241-5_amd64.deb: 404  Not Found [IP: 208.80.154.15 80]

Looking at the host integration-slave-jessie-1004.integration.eqiad.wmflabs and trying to run the update:

# crontab -l|grep buster
# Puppet Name: cowbuilder_update_buster-amd64
34 7 * * * /usr/sbin/cowbuilder --update                     --basepath "/srv/pbuilder/base-buster-amd64.cow"                     >/dev/null 2>&1
#  /usr/sbin/cowbuilder --update --basepath "/srv/pbuilder/base-buster-amd64.cow"
 -> Copying COW directory
  forking: rm -rf /var/cache/pbuilder/build//cow.28076 
  forking: cp -al /srv/pbuilder/base-buster-amd64.cow /var/cache/pbuilder/build//cow.28076 
I: removed stale ilistfile /var/cache/pbuilder/build//cow.28076/.ilist
 -> Invoking pbuilder
  forking: pbuilder update --buildplace /var/cache/pbuilder/build//cow.28076 --no-targz --internal-chrootexec chroot /var/cache/pbuilder/build//cow.28076 cow-shell 
W: /root/.pbuilderrc does not exist
I: Running in no-targz mode
I: Current time: Tue Oct 29 14:24:27 UTC 2019
I: pbuilder-time-stamp: 1572359067
I: copying local configuration
I: mounting /proc filesystem
I: mounting /run/shm filesystem
I: mounting /dev/pts filesystem
I: policy-rc.d already exists
I: Refreshing the base.tgz 
I: upgrading packages
Get:1 http://mirrors.wikimedia.org/debian buster InRelease [122 kB]
Reading package lists...
E: Repository 'http://mirrors.wikimedia.org/debian buster InRelease' changed its 'Suite' value from 'testing' to 'stable'
I: unmounting dev/pts filesystem
I: unmounting run/shm filesystem
I: unmounting proc filesystem
pbuilder update failed
E: could not update with cowdancer, try --no-cowdancer-update option
  forking: rm -rf /var/cache/pbuilder/build//cow.28076
E: Repository 'http://mirrors.wikimedia.org/debian buster InRelease' changed its 'Suite' value from 'testing' to 'stable'

I guess because the cowimage has been created before buster got cut as stable?

Event Timeline

# head /var/cache/pbuilder/base-buster-amd64.cow/var/lib/apt/lists/mirrors.wikimedia.org_debian_dists_buster_InRelease
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Origin: Debian
Label: Debian
Suite: testing
Codename: buster
Changelogs: http://metadata.ftp-master.debian.org/changelogs/@CHANGEPATH@_changelog
Date: Sat, 06 Jul 2019 02:26:45 UTC
Valid-Until: Sat, 13 Jul 2019 02:26:45 UTC

Mentioned in SAL (#wikimedia-releng) [2019-10-29T14:34:08Z] <hashar> integration-slave-jessie-* : rm /var/cache/pbuilder/base-buster-amd64.cow/var/lib/apt/lists/mirrors.wikimedia.org_debian_dists_buster_InRelease && /usr/sbin/cowbuilder --update --basepath "/srv/pbuilder/base-buster-amd64.cow" # T236796

hashar claimed this task.

I have just moved the file:

# mv /var/cache/pbuilder/base-buster-amd64.cow/var/lib/apt/lists/mirrors.wikimedia.org_debian_dists_buster_InRelease \
     /var/cache/pbuilder/base-buster-amd64.cow/var/lib/apt/lists/mirrors.wikimedia.org_debian_dists_buster_InRelease-back

It works now.