Page MenuHomePhabricator

Update Debian package for Blubber
Closed, ResolvedPublic

Description

upstream v0.2.0 has been pushed, debian/0.2.0 has been tagged and debian/changelog has been updated on the debian branch. Should just need a rebuild for the debian repositories.

Revisions and Commits

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 7 2017, 9:24 PM
thcipriani added subscribers: dduvall, akosiaris, Joe.

adding @akosiaris and @Joe since they have the most background on Blubber: could one of you upload a new version of blubber to the apt repository/let me or @dduvall know if there are changes needed?

debian/changelog in that package is wrongly formatted and hence package is currently unbuildable. See D875

MoritzMuehlenhoff triaged this task as Medium priority.Nov 13 2017, 7:35 AM

debian/changelog in that package is wrongly formatted and hence package is currently unbuildable. See D875

I landed your change in D875 and attempted to build the package, but it seems like there may be a few BuildDependencies that are not packaged. The github.com/docker/distribution/reference dependency is probably one we will just need to work around. The other dependency gopkg.in/go-playground/validator.v9 isn't packaged. but gopkg.in/go-playground/validator.v8 is. We can probably work with that package; however, I noticed it's only packaged for testing and unstable (https://packages.debian.org/search?keywords=golang-gopkg-go-playground-validator.v8-dev). Is that fine for builddependencies?

How do you plan on working around github.com/docker/distribution/reference. If it's vendoring [1] then might as well do it for gopkg.in/go-playground/validator.v9 as well, since backporting it from stretch/unstable can prove to be more difficult than it's worth it

[1] https://blog.gopheracademy.com/advent-2015/vendor-folder/

I 've tried building the package once more. It fails as it needs a tag for 00820cbd6bbcc98321c5a0d279394673425d0783 (I am guessing 0.2.1 ?) and a corresponding debian/changelog entry for it as well

Since we hadn't actually released 0.2.0 and there weren't any changes other than Debian package related ones, I moved the existing v0.2.0 tag, created new ones for debian/0.1.0-1 and debian/0.2.0-1, and deleted debian/0.1.0 and debian/0.2.0. In retrospect maybe wasn't the best idea... but you should be able to delete your local tags and pull them down again from diffusion.

blubber $ git ls-remote --tags
From ssh://vcs@git-ssh.wikimedia.org/source/blubber.git
8ff4d41552d527eec484a07b76d764e52559e340	refs/tags/debian/0.0_git20170911.0.4e5c728-1
8a348baa7063666644e726e3970eb22d6042d302	refs/tags/debian/0.1.0-1
72fdd23509b52f04d14f25aba821399289fc7bc9	refs/tags/debian/0.2.0-1
5d4cc80edf4fdba60069a7ef1e450117195fb987	refs/tags/v0.1.0
00820cbd6bbcc98321c5a0d279394673425d0783	refs/tags/v0.2.0

OK, I git pulled and refreshed tags. Unfortunately we are still at a no-go state. I now have

dh_auto_build -O--buildsystem=golang -- -ldflags "-X phabricator.wikimedia.org/source/blubber/meta.Version=0.2.0 -X phabricator.wikimedia.org/source/blubber/meta.GitCommit="
	go install -v -p 8 -ldflags "-X phabricator.wikimedia.org/source/blubber/meta.Version=0.2.0 -X phabricator.wikimedia.org/source/blubber/meta.GitCommit=" phabricator.wikimedia.org/source/blubber phabricator.wikimedia.org/source/blubber/build phabricator.wikimedia.org/source/blubber/config phabricator.wikimedia.org/source/blubber/docker phabricator.wikimedia.org/source/blubber/meta phabricator.wikimedia.org/source/blubber/vendor/github.com/davecgh/go-spew/spew phabricator.wikimedia.org/source/blubber/vendor/github.com/docker/distribution phabricator.wikimedia.org/source/blubber/vendor/github.com/docker/distribution/digest phabricator.wikimedia.org/source/blubber/vendor/github.com/docker/distribution/reference phabricator.wikimedia.org/source/blubber/vendor/github.com/go-playground/locales phabricator.wikimedia.org/source/blubber/vendor/github.com/go-playground/locales/cu phabricator.wikimedia.org/source/blubber/vendor/github.com/go-playground/locales/currency phabricator.wikimedia.org/source/blubber/vendor/github.com/go-playground/universal-translator phabricator.wikimedia.org/source/blubber/vendor/github.com/pmezard/go-difflib/difflib phabricator.wikimedia.org/source/blubber/vendor/github.com/stretchr/testify phabricator.wikimedia.org/source/blubber/vendor/github.com/stretchr/testify/assert phabricator.wikimedia.org/source/blubber/vendor/gopkg.in/go-playground/validator.v9 phabricator.wikimedia.org/source/blubber/vendor/gopkg.in/yaml.v2
src/phabricator.wikimedia.org/source/blubber/vendor/github.com/docker/distribution/blobs.go:10:2: cannot find package "github.com/docker/distribution/context" in any of:
	/build/blubber-0.2.0/obj-x86_64-linux-gnu/src/phabricator.wikimedia.org/source/blubber/vendor/github.com/docker/distribution/context (vendor tree)
	/usr/lib/go-1.7/src/github.com/docker/distribution/context (from $GOROOT)
	/build/blubber-0.2.0/obj-x86_64-linux-gnu/src/github.com/docker/distribution/context (from $GOPATH)
src/phabricator.wikimedia.org/source/blubber/vendor/github.com/stretchr/testify/doc.go:19:2: cannot find package "github.com/stretchr/testify/http" in any of:
	/build/blubber-0.2.0/obj-x86_64-linux-gnu/src/phabricator.wikimedia.org/source/blubber/vendor/github.com/stretchr/testify/http (vendor tree)
	/usr/lib/go-1.7/src/github.com/stretchr/testify/http (from $GOROOT)
	/build/blubber-0.2.0/obj-x86_64-linux-gnu/src/github.com/stretchr/testify/http (from $GOPATH)
src/phabricator.wikimedia.org/source/blubber/vendor/github.com/stretchr/testify/doc.go:21:2: cannot find package "github.com/stretchr/testify/mock" in any of:
	/build/blubber-0.2.0/obj-x86_64-linux-gnu/src/phabricator.wikimedia.org/source/blubber/vendor/github.com/stretchr/testify/mock (vendor tree)
	/usr/lib/go-1.7/src/github.com/stretchr/testify/mock (from $GOROOT)
	/build/blubber-0.2.0/obj-x86_64-linux-gnu/src/github.com/stretchr/testify/mock (from $GOPATH)
dh_auto_build: go install -v -p 8 -ldflags -X phabricator.wikimedia.org/source/blubber/meta.Version=0.2.0 -X phabricator.wikimedia.org/source/blubber/meta.GitCommit= phabricator.wikimedia.org/source/blubber phabricator.wikimedia.org/source/blubber/build phabricator.wikimedia.org/source/blubber/config phabricator.wikimedia.org/source/blubber/docker phabricator.wikimedia.org/source/blubber/meta phabricator.wikimedia.org/source/blubber/vendor/github.com/davecgh/go-spew/spew phabricator.wikimedia.org/source/blubber/vendor/github.com/docker/distribution phabricator.wikimedia.org/source/blubber/vendor/github.com/docker/distribution/digest phabricator.wikimedia.org/source/blubber/vendor/github.com/docker/distribution/reference phabricator.wikimedia.org/source/blubber/vendor/github.com/go-playground/locales phabricator.wikimedia.org/source/blubber/vendor/github.com/go-playground/locales/cu phabricator.wikimedia.org/source/blubber/vendor/github.com/go-playground/locales/currency phabricator.wikimedia.org/source/blubber/vendor/github.com/go-playground/universal-translator phabricator.wikimedia.org/source/blubber/vendor/github.com/pmezard/go-difflib/difflib phabricator.wikimedia.org/source/blubber/vendor/github.com/stretchr/testify phabricator.wikimedia.org/source/blubber/vendor/github.com/stretchr/testify/assert phabricator.wikimedia.org/source/blubber/vendor/gopkg.in/go-playground/validator.v9 phabricator.wikimedia.org/source/blubber/vendor/gopkg.in/yaml.v2 returned exit code 1

Which means it complains about not finding https://github.com/docker/distribution. Which is weird because it's supposed to be vendored-in anyway. Looking at https://github.com/docker/distribution I see the context package there. Is it possible that during the vendoring, not everything was pulled in ?

Which means it complains about not finding https://github.com/docker/distribution. Which is weird because it's supposed to be vendored-in anyway. Looking at https://github.com/docker/distribution I see the context package there. Is it possible that during the vendoring, not everything was pulled in ?

FWIW I was able to build this package with golang 1.9.2 on buster. I would assume that this is dep being "smart" somewhere along the way. I don't know if we ran dep prune but I remember @dduvall mentioning something along those lines...If so there is a somewhat ominous warning there: https://github.com/golang/dep/blob/master/cmd/dep/prune.go#L28-L29

Which means it complains about not finding https://github.com/docker/distribution. Which is weird because it's supposed to be vendored-in anyway. Looking at https://github.com/docker/distribution I see the context package there. Is it possible that during the vendoring, not everything was pulled in ?

FWIW I was able to build this package with golang 1.9.2 on buster. I would assume that this is dep being "smart" somewhere along the way.

Gah! This indeed works. I have no idea why on earth the package would be unbuildable on go 1.7 but buildable on go 1.9 while the error is about a missing dependency. My Go foo is not that strong yet. Anyway I 've built the package on buster and uploaded to jessie-wikimedia. The only reason this will work is because it's go and has very very few dynamic linking dependencies. I 've already done the upgrade on contint1001. I am guessing we can resolve this, but if you have any info about the version puzzle, please share.

I am guessing we can resolve this, but if you have any info about the version puzzle, please share.

I think this has to do with the change in wildcard (...) matching in golang 1.9 (https://golang.org/doc/go1.9#vendor-dotdotdot). Since vendor is excluded from wildcard matching in golang 1.9 you see:

$ go list phabricator.wikimedia.org/source/blubber/...
phabricator.wikimedia.org/source/blubber
phabricator.wikimedia.org/source/blubber/build
phabricator.wikimedia.org/source/blubber/config
phabricator.wikimedia.org/source/blubber/docker
phabricator.wikimedia.org/source/blubber/meta

Whereas with golang 1.7.4 (on stretch) you see:

$ go list  phabricator.wikimedia.org/source/blubber/...
phabricator.wikimedia.org/source/blubber
phabricator.wikimedia.org/source/blubber/build
phabricator.wikimedia.org/source/blubber/config
phabricator.wikimedia.org/source/blubber/docker
phabricator.wikimedia.org/source/blubber/meta
phabricator.wikimedia.org/source/blubber/vendor/github.com/davecgh/go-spew/spew
phabricator.wikimedia.org/source/blubber/vendor/github.com/docker/distribution
phabricator.wikimedia.org/source/blubber/vendor/github.com/docker/distribution/digest
phabricator.wikimedia.org/source/blubber/vendor/github.com/docker/distribution/reference
phabricator.wikimedia.org/source/blubber/vendor/github.com/go-playground/locales
phabricator.wikimedia.org/source/blubber/vendor/github.com/go-playground/locales/cu
phabricator.wikimedia.org/source/blubber/vendor/github.com/go-playground/locales/currency
phabricator.wikimedia.org/source/blubber/vendor/github.com/go-playground/universal-translator
phabricator.wikimedia.org/source/blubber/vendor/github.com/pmezard/go-difflib/difflib
phabricator.wikimedia.org/source/blubber/vendor/github.com/stretchr/testify
phabricator.wikimedia.org/source/blubber/vendor/github.com/stretchr/testify/assert
phabricator.wikimedia.org/source/blubber/vendor/gopkg.in/go-playground/validator.v9
phabricator.wikimedia.org/source/blubber/vendor/gopkg.in/yaml.v2

We may be able to work around this using DH_GOLANG_EXCLUDES in the debian/rules file (this seems like what was done for git lfs https://github.com/git-lfs/git-lfs/blob/master/debian/rules#L20)

akosiaris added a comment.EditedNov 27 2017, 10:12 AM

The DH_GOLANG_EXCLUDES seems to have worked. I was successfully able to build the package on stretch as well. @thcipriani Nice! Thanks for researching this further!