pmacct should be upgraded to 1.6.2 on Stretch
Closed, ResolvedPublic
Actions

Assigned To

None

Authored By

	elukey
	Aug 17 2017, 8:52 AM

Description

In https://phabricator.wikimedia.org/T172681 we discovered that the combination of pmacct + librdkafka 0.11 on rhenium was causing problems to the Analytics Kafka brokers.

I tried to fix the problem with https://gerrit.wikimedia.org/r/371937 but I forgot to verify if it was supported by the current Stretch version, and of course it is not (1.6.1).

The 1.6.2 version seems supporting librdkafka settings, so it would be enough to package/deploy 1.6.2 on
rhenium.

Opened a Debian bug: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=872327

I masked the related unit on rhenium and disabled puppet.

Details

	Subject	Repo	Branch	Lines +/-
	profile::pmacct: move configuration to Kafka Jumbo	operations/puppet	production	+26 -14
	profile::pmacct: pin librdkafka to stretch version	operations/puppet	production	+11 -0

Customize query in gerrit

Related Objects

Mentioned In: T178432: Update librdkafka on banner log boxen

Event Timeline

elukey created this task.Aug 17 2017, 8:52 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 17 2017, 8:52 AM

elukey updated the task description. (Show Details)Aug 17 2017, 8:55 AM

ayounsi subscribed.Aug 21 2017, 11:29 PM

As Faidon pointed out in the other task, it is stretch-backports that offers librdkafka 0.11, not the regular stretch repo, so a quick solution for rhenium would be to pin librdkafka to the stretch version (if possible).

elukey added a project: User-Elukey.Aug 28 2017, 4:32 PM

Change 374360 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] profile::pmacct: pin librdkafka to stretch version

https://gerrit.wikimedia.org/r/374360

gerritbot added a project: Patch-For-Review.Aug 28 2017, 5:16 PM

Mentioned in SAL (#wikimedia-operations) [2017-08-29T09:29:13Z] <elukey> re-installed pmacct/librdkafka1/kafkacat on rhenium with stretch versions - T173489

Change 374360 abandoned by Elukey:
profile::pmacct: pin librdkafka to stretch version

https://gerrit.wikimedia.org/r/374360

elukey moved this task from Backlog to Ops Backlog on the User-Elukey board.Sep 12 2017, 10:45 AM

Jgreen mentioned this in T178432: Update librdkafka on banner log boxen.Oct 17 2017, 8:53 PM

pmacct 1.7.0-1 (with GeoIP2 support too!) was uploaded to sid yesterday. This should be as easy as a backport-and-install now.

Could be a good candidate for the Kafka Jumbo cluster! In this case it could use librdkafka 0.11 to negotiate API without causing troubles :)

If pmacct, as I imagine, doesn't carry any sensitive data that needs to be restricted, we could directly use the plaintext port (9092) of kafka-jumbo100[1-6]

Change 392007 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] profile::pmacct: move configuration to Kafka Jumbo

https://gerrit.wikimedia.org/r/392007

pmacct 1.7 should not be strictly needed now that we have a new Kafka 0.11 cluster (jumbo)!

Change 392007 merged by Elukey:
[operations/puppet@production] profile::pmacct: move configuration to Kafka Jumbo