Page MenuHomePhabricator

Build Kafka 0.8.1.1 package for Jessie and upgrade Brokers to Jessie.
Closed, DeclinedPublic

Description

We can't use Precise if we want to support IPSec on Kafka brokers. We should upgrade these machines to Jessie. Let's not try to upgrade Kafka versions as part of this ticket.

Event Timeline

Ottomata created this task.May 5 2015, 4:15 PM
Ottomata claimed this task.
Ottomata raised the priority of this task from to Needs Triage.
Ottomata updated the task description. (Show Details)
Ottomata added subscribers: Ottomata, Gage, JAllemandou.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 5 2015, 4:15 PM
Gage added a comment.EditedMay 19 2015, 3:04 AM

I followed this: https://git.wikimedia.org/blob/operations%2Fpuppet.git/2cdd08f9686b040816bd0dd8e63e712f4b084a7a/modules%2Fpackage_builder%2FREADME.md

So I tried this:

gage@copper:~/src/kafka$ GIT_PBUILDER_AUTOCONF=no DIST=jessie WIKIMEDIA=yes git-buildpackage -us -uc --git-builder=git-pbuilder

But I got this:

1 packages upgraded, 178 newly installed, 0 to remove and 13 not upgraded.
Need to get 113 MB/133 MB of archives. After unpacking 264 MB will be used.
The following packages have unmet dependencies:
 pbuilder-satisfydepends-dummy : Depends: libcglib-java which is a virtual package.
                                 Depends: libcommons-collections-java which is a virtual package.
                                 Depends: libparanamer-java which is a virtual package.
                                 Depends: libsnappy-java which is a virtual package.
                                 Depends: libxmlenc-java which is a virtual package.
Unable to resolve dependencies!  Giving up...

So do we build all of these deps for Jessie? Or use Archiva?

Gage set Security to None.May 19 2015, 3:05 AM
Gage added a subscriber: akosiaris.

I think we should either backport or stick them manually in lib/. I don't think we should use Archiva for building this if we don't have to.

faidon added a subscriber: faidon.Jun 2 2015, 5:57 PM

Yeah, what @Ottomata said, it's just a handful of packages and we have this running on Ubuntu as well, so how hard can it be... On the above:

  • libcglib-java is available on sid. Probably installable as-is on jessie.
  • libcommons-collections-java - replaced by libcommons-collections3-java, not sure if Kafka uses that. #744226
  • libparanamer-java also available on sid. Was not part of jessie because of #776583
  • libsnappy-java, available in sid/stretch. Probably backportable very easily.
  • libxmlenc-java is currently sitting in NEW. We can rebuild from git.

Also note that there's an ITP for Kafka at Debian as well, #786460 with a proposed package already; we may soon not have to rely on our own work. @akosiaris, you might want to contribute your work back there too.

Restricted Application added a subscriber: Matanya. · View Herald TranscriptJul 22 2015, 10:09 PM

I was able to build a 0.8.1.1 Jessie package using Alex's debianization. I did it by adding the noted missing dependency jars to ext/ and changing CLASSPATHs and Build-Depends, and adding them to source/include-binaries.

I could attempt to bring these dependencies into our apt, but I'm not sure it is worth it yet, seeing as I now want to upgrade to 0.8.2.1 soon. (This has become a dependency for some EventLogging work.) We'll have to completely re-work the packaging for 0.8.2.1 since it uses Gradle, and who knows what we'll have to do to make that work.

Faidon, would you be ok if I proceeded this way, or should I look into bringing these deps in as .debs?

Ottomata closed this task as Declined.Jul 31 2015, 2:09 PM

I am declining this ticket in favor of skipping right to Jessie + 0.8.2.1 T106581.