pmacct should be upgraded to 1.6.2 on Stretch
Open, NormalPublic

Description

In https://phabricator.wikimedia.org/T172681 we discovered that the combination of pmacct + librdkafka 0.11 on rhenium was causing problems to the Analytics Kafka brokers.

I tried to fix the problem with https://gerrit.wikimedia.org/r/371937 but I forgot to verify if it was supported by the current Stretch version, and of course it is not (1.6.1).

The 1.6.2 version seems supporting librdkafka settings, so it would be enough to package/deploy 1.6.2 on
rhenium.

Opened a Debian bug: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=872327

I masked the related unit on rhenium and disabled puppet.

elukey created this task.Aug 17 2017, 8:52 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 17 2017, 8:52 AM
elukey updated the task description. (Show Details)Aug 17 2017, 8:55 AM

As Faidon pointed out in the other task, it is stretch-backports that offers librdkafka 0.11, not the regular stretch repo, so a quick solution for rhenium would be to pin librdkafka to the stretch version (if possible).

Change 374360 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] profile::pmacct: pin librdkafka to stretch version

https://gerrit.wikimedia.org/r/374360

Mentioned in SAL (#wikimedia-operations) [2017-08-29T09:29:13Z] <elukey> re-installed pmacct/librdkafka1/kafkacat on rhenium with stretch versions - T173489

Change 374360 abandoned by Elukey:
profile::pmacct: pin librdkafka to stretch version

https://gerrit.wikimedia.org/r/374360

elukey moved this task from Backlog to Ops Backlog on the User-Elukey board.Sep 12 2017, 10:45 AM

pmacct 1.7.0-1 (with GeoIP2 support too!) was uploaded to sid yesterday. This should be as easy as a backport-and-install now.

elukey added a comment.EditedMon, Oct 23, 4:00 PM

Could be a good candidate for the Kafka Jumbo cluster! In this case it could use librdkafka 0.11 to negotiate API without causing troubles :)

If pmacct, as I imagine, doesn't carry any sensitive data that needs to be restricted, we could directly use the plaintext port (9092) of kafka-jumbo100[1-6]

Change 392007 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] profile::pmacct: move configuration to Kafka Jumbo

https://gerrit.wikimedia.org/r/392007

pmacct 1.7 should not be strictly needed now that we have a new Kafka 0.11 cluster (jumbo)!