labs/toollabs should always run debian-glue, possibly with multiple distributions
Open, Needs TriagePublic

Description

Follow up to T210663

There are a few CI problems for labs/toollabs.git. The repository is a Debian native package intended to be run on Trusty, Jessie and Stretch.

  • as a Debian native package, we should probably run the debian glue job even when no file is changed in ./debian
  • the debian glue job uses the distro from the changelog file which is unstable. It might be helpful to build for trusty, jessie and stretch.
hashar created this task.Nov 29 2018, 7:56 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 29 2018, 7:56 PM
hashar added a subscriber: aborrero.

context

For repositories having a Debian package, we run the Jenkins job debian-glue. It is a wrapper around git buildpackage/cowbuilder and get the target distribution from the debian/changelog file. Hence it builds for a single distribution.

The job only triggers when a file under debian/ directory is altered.

The CI configuration for labs/toollabs looks like:

zuul/layout.yaml
- name: labs/toollabs
  template:
    - name: tox-docker
    - name: php55lint
    - name: php70lint
  test:
   - debian-glue
  gate-and-submit:
   - debian-glue

tox-docker runs tox under Debian Stretch with python 2.7, 3.4, 3.5, 3.6, 3.7 available. It always trigger.

fix up

For multiple distributions, we need a new set of debian-glue jobs that come with an hardcoded distribution instead of guessing it from the debian/changelog.

Being a native package, ideally the jobs should always triggered, even when no files is changed in debian/ directory. Though probably it is easy enough to just add an entry to debian/changelog whenever doing a patch. It is not ideal, but simpler CI wise though jobs would NOT be triggered if someone forget to bump the changelog. That can be figured out later though.

hashar claimed this task.Wed, Jan 2, 9:57 AM

The idea comes from https://gerrit.wikimedia.org/r/#/c/labs/toollabs/+/476092/ which fails on CI. That is due to the changelog pointing at unstable and python 3.7 got introduced at some point and thus breaking the build.

I am fine targeting unstable. Since the master branch is intended to be for multiple distributions, we surely need to build the labs/toollabs Debian package for all the distributions it supports.

Change 481670 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] debian-glue jobs with hardcoded distributions

https://gerrit.wikimedia.org/r/481670

Mentioned in SAL (#wikimedia-releng) [2019-01-02T10:19:32Z] <hashar> updating all debian-glue jobs and creating new ones with hardcoded distributions (trusty, jessie, stretch, unstable) T210780

Change 481670 merged by jenkins-bot:
[integration/config@master] debian-glue jobs with hardcoded dists for labs/toollabs

https://gerrit.wikimedia.org/r/481670

Change 481835 had a related patch set uploaded (by Hashar; owner: Hashar):
[labs/toollabs@master] Dummy build to exercise Wikimeda CI

https://gerrit.wikimedia.org/r/481835

I have send a dummy change in Gerrit that just bump the changelog (#481835):

Job nameStatusDurationVoting?
debian-glue-unstableSUCCESS40svoting
debian-glue-trusty-non-votingFAILURE44snon-voting
debian-glue-jessie-non-votingFAILURE35snon-voting
debian-glue-stretch-non-votingSUCCESS32snon-voting

The failures on Trusty and Jessie are due to lintian. Test results:

  • https://integration.wikimedia.org/ci/job/debian-glue-trusty-non-voting/1/testReport/
    • E: toollabs changes: bad-distribution-in-changes-file unstable
      • Some how CI creates a dummy changelog, I guess it does not recognizes trusty and fallback to unstable :/
    • E: toollabs source: missing-build-dependency-for-dh-addon python3 => python3 | python3-all | python3-dev | python3-all-dev
      • The package has a build dependency set to python3:any which apparently is not recognized by the lintian version in Trusty?

For Trusty the dsc has:

Changes: 
 toollabs (1.35+nmu1~20190102102533.1+trusty) unstable; urgency=medium
 .
   [ Antoine Musso ]
   * Non-maintainer upload.
   * Dummy build for CI - T210780
 .
   [ jenkins-debian-glue Autobuilder ]
   * SNAPSHOT autobuild for unreleased 1.35+nmu1 via jenkins-debian-glue.
aborrero added a comment.EditedWed, Jan 2, 12:19 PM

The d/changelog file actually has unstable as distribution. Also, the lintian warning refers to the .changes file, generated after the package build

Not sure about git buildpackage/cowbuilder but with sbuild you can specify for which distro you want to build the package independent of what d/changelog says (the -d option).
That will set the Distribution field in the .changes file to what you want.
There is another way to workaround this, and is by using lintian profiles:

It is possible that you are uploading for a different distribution than the one Lintian is checking for. In that case, passing --profile $VENDOR may fix this warning.

Not sure how much work on your side could mean any of the solutions.

  • E: toollabs source: missing-build-dependency-for-dh-addon python3 => python3 | python3-all | python3-dev | python3-all-dev
    • The package has a build dependency set to python3:any which apparently is not recognized by the lintian version in Trusty?

This usually means the package requires a build-dep on dh-python. I don't think this error is induced by the CI system.

Same. This usually means the package requires a build-dep on dh-python. I don't think this error is induced by the CI system.

hashar added a comment.Wed, Jan 2, 3:44 PM

Not sure about git buildpackage/cowbuilder but with sbuild you can specify for which distro you want to build the package independent of what d/changelog says (the -d option).
That will set the Distribution field in the .changes file to what you want.

The CI jobs use the cow images set by the puppet module package_buider wich uses pbuilder/cowbuilder. There is a wrapping shell script Jenkins Debian Glue which uses git buildpackage on top of that :)

I eventually found the issue which is that the Jenkins Debian Glue script generate-git-snapshot does not set the distribution when creating the source changes (THANKS for the hint!). The DIST environment variable has to be set which is then passed as git buildpackage --distribution=$DIST.

I have set DIST and the .changes is correct. A side effect is the source changes is kept around (as <foo>_source.changes).

I have confirmed the Distribution field is properly set. Eg jessie-wikimedia or trusty. Thus the lintian error E: toollabs changes: bad-distribution-in-changes-file unstable is gone!

Change 481870 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] debian-glue fix Distribution in .changes file

https://gerrit.wikimedia.org/r/481870

Change 481870 merged by jenkins-bot:
[integration/config@master] debian-glue fix Distribution in .changes file

https://gerrit.wikimedia.org/r/481870

hashar added a comment.Wed, Jan 2, 4:23 PM

There is another way to workaround this, and is by using lintian profiles:

It is possible that you are uploading for a different distribution than the one Lintian is checking for. In that case, passing --profile $VENDOR may fix this warning.

We do not use lintian profiles per see. lintian is just run inside the same cowbuilder to use the version from the target distribution.


For Jessie and Trusty lintian causes:

E: toollabs source: missing-build-dependency-for-dh-addon python3 => python3 | python3-all | python3-dev | python3-all-dev

I did a bit of digging, labs/toollabs has Build-Depends: python3:any and dh-python in Jessie/Stretch depends on python3-minimal. It looked like python3:any did not recognizes python3-minimal.

Eventually I have found https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=803164 lintian: Bogus missing-python-build-dependency and missing-build-dependency-for-dh-addon which should be fixed in Lintian 2.5.44 (Jessie has 2.5.30).

  • data/debhelper/dh_addons-manual: + [NT] Add ":any" to the dependencies for the python add-ons. Thanks to Barry Warsaw for reporting the bug. (Closes: #803164)

That leads to lintian commit 8d53737 - Add missing ":any" to some python relations.

We might be able to request the Jessie version to be fixed, though that is probably unlikely to happen. Maybe we can instead add a lintian override?

Change 481876 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] labs/toollabs .deb works for Stretch

https://gerrit.wikimedia.org/r/481876