We can't use Precise if we want to support IPSec on Kafka brokers. We should upgrade these machines to Jessie. Let's not try to upgrade Kafka versions as part of this ticket.
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | • kevinator | T102225 {stag} EventLogging on Kafka | |||
| Resolved | Ottomata | T102831 Prep work for Eventlogging on Kafka {stag} | |||
| Resolved | Ottomata | T104228 Puppetize parallel eventlogging-processor {stag} [5 pts] | |||
| Resolved | LSobanski | T111653 Encrypt all the things | |||
| Resolved | BBlack | T92602 Secure inter-datacenter web request log (Kafka) traffic | |||
| Resolved | Ottomata | T106581 Build 0.8.2.1 Kafka package and upgrade Kafka brokers | |||
| Declined | Ottomata | T98161 Build Kafka 0.8.1.1 package for Jessie and upgrade Brokers to Jessie. | |||
| Resolved | Ottomata | T103106 Create jmxtrans Jessie package |
Event Timeline
Comment Actions
I followed this: https://git.wikimedia.org/blob/operations%2Fpuppet.git/2cdd08f9686b040816bd0dd8e63e712f4b084a7a/modules%2Fpackage_builder%2FREADME.md
So I tried this:
gage@copper:~/src/kafka$ GIT_PBUILDER_AUTOCONF=no DIST=jessie WIKIMEDIA=yes git-buildpackage -us -uc --git-builder=git-pbuilder
But I got this:
1 packages upgraded, 178 newly installed, 0 to remove and 13 not upgraded.
Need to get 113 MB/133 MB of archives. After unpacking 264 MB will be used.
The following packages have unmet dependencies:
pbuilder-satisfydepends-dummy : Depends: libcglib-java which is a virtual package.
Depends: libcommons-collections-java which is a virtual package.
Depends: libparanamer-java which is a virtual package.
Depends: libsnappy-java which is a virtual package.
Depends: libxmlenc-java which is a virtual package.
Unable to resolve dependencies! Giving up...- https://packages.debian.org/search?keywords=libcglib-java
- On Wheezy and Stretch but not Jessie.
- https://packages.debian.org/search?keywords=libcommons-collections-java
- Only on Wheezy
- https://packages.debian.org/search?keywords=libparanamer-java
- On Squeeze and Sid, but not Jessie
- https://packages.debian.org/search?keywords=libsnappy-java
- On Wheezy and Stretch, but not Jessie.
- https://packages.debian.org/search?keywords=libxmlenc-java
- Only on Squeeze
So do we build all of these deps for Jessie? Or use Archiva?
Comment Actions
I think we should either backport or stick them manually in lib/. I don't think we should use Archiva for building this if we don't have to.
Comment Actions
Yeah, what @Ottomata said, it's just a handful of packages and we have this running on Ubuntu as well, so how hard can it be... On the above:
- libcglib-java is available on sid. Probably installable as-is on jessie.
- libcommons-collections-java - replaced by libcommons-collections3-java, not sure if Kafka uses that. #744226
- libparanamer-java also available on sid. Was not part of jessie because of #776583
- libsnappy-java, available in sid/stretch. Probably backportable very easily.
- libxmlenc-java is currently sitting in NEW. We can rebuild from git.
Also note that there's an ITP for Kafka at Debian as well, #786460 with a proposed package already; we may soon not have to rely on our own work. @akosiaris, you might want to contribute your work back there too.
Comment Actions
I was able to build a 0.8.1.1 Jessie package using Alex's debianization. I did it by adding the noted missing dependency jars to ext/ and changing CLASSPATHs and Build-Depends, and adding them to source/include-binaries.
I could attempt to bring these dependencies into our apt, but I'm not sure it is worth it yet, seeing as I now want to upgrade to 0.8.2.1 soon. (This has become a dependency for some EventLogging work.) We'll have to completely re-work the packaging for 0.8.2.1 since it uses Gradle, and who knows what we'll have to do to make that work.
Faidon, would you be ok if I proceeded this way, or should I look into bringing these deps in as .debs?