Page MenuHomePhabricator

Upgrade librdkafka on eventlog1002
Closed, ResolvedPublic3 Story Points

Description

While working on the last Eventlogging outage, I noticed that we are running a very old version of librdkafka on eventlog1002:

elukey@eventlog1002:~$ dpkg -l | grep kafka
ii  kafkacat                                  1.3.0-1+b1                     amd64        generic producer and consumer for Apache Kafka
ii  librdkafka1:amd64                         0.9.3-1                        amd64        library implementing the Apache Kafka protocol
ii  python-confluent-kafka                    0.9.2-1                        amd64        Python client to interact with Kafka - Python 2.7
ii  python-kafka                              1.4.1-1~stretch1               all          Pure Python client for Apache Kafka
ii  python-pykafka                            2.4.0-1~stretch1               amd64        Full-Featured Pure-Python Kafka Client

IIUC confluent-kafka-python uses librdkafka, if so it would be great to deploy the last version in stretch-backports (0.11.5) and possibly package/deploy the latest confluent-kafka-python as well (0.11.5).

Event Timeline

elukey triaged this task as Normal priority.Jul 31 2018, 9:28 AM
elukey created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 31 2018, 9:28 AM
Ottomata claimed this task.Aug 15 2018, 5:31 PM
Ottomata set the point value for this task to 3.

Change 452958 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Use newer librdkafka for eventlogging

https://gerrit.wikimedia.org/r/452958

Currently running librdkafka1 0.11.5-1~bpo9+1 in deployment-prep. Will leave it for a day or two and then upgrade in prod.

Ottomata moved this task from Next Up to In Progress on the Analytics-Kanban board.

Change 452958 merged by Ottomata:
[operations/puppet@production] Use newer librdkafka for eventlogging

https://gerrit.wikimedia.org/r/452958

Change 454074 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Fix librdkafka backports pin for eventlogging

https://gerrit.wikimedia.org/r/454074

Change 454074 merged by Ottomata:
[operations/puppet@production] Fix librdkafka backports pin for eventlogging

https://gerrit.wikimedia.org/r/454074

Mentioned in SAL (#wikimedia-analytics) [2018-08-20T18:18:34Z] <ottomata> restaring eventlogging client side processes using librdkafka1 0.11.x - https://phabricator.wikimedia.org/T200769

Nuria closed this task as Resolved.Aug 30 2018, 12:30 AM