Page MenuHomePhabricator
Create Task
Maniphest T200769

Upgrade librdkafka on eventlog1002
Closed, ResolvedPublic3 Estimated Story Points
Actions

Description

While working on the last Eventlogging outage, I noticed that we are running a very old version of librdkafka on eventlog1002:

elukey@eventlog1002:~$ dpkg -l | grep kafka
ii  kafkacat                                  1.3.0-1+b1                     amd64        generic producer and consumer for Apache Kafka
ii  librdkafka1:amd64                         0.9.3-1                        amd64        library implementing the Apache Kafka protocol
ii  python-confluent-kafka                    0.9.2-1                        amd64        Python client to interact with Kafka - Python 2.7
ii  python-kafka                              1.4.1-1~stretch1               all          Pure Python client for Apache Kafka
ii  python-pykafka                            2.4.0-1~stretch1               amd64        Full-Featured Pure-Python Kafka Client

IIUC confluent-kafka-python uses librdkafka, if so it would be great to deploy the last version in stretch-backports (0.11.5) and possibly package/deploy the latest confluent-kafka-python as well (0.11.5).

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Fix librdkafka backports pin for eventloggingoperations/puppetproduction+0 -1
Use newer librdkafka for eventloggingoperations/puppetproduction+10 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

elukey created this task.Jul 31 2018, 9:28 AM
elukey triaged this task as Medium priority.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 31 2018, 9:28 AM
Milimetric moved this task from Incoming to Operational Excellence on the Analytics board.Aug 2 2018, 3:32 PM
Ottomata claimed this task.Aug 15 2018, 5:31 PM
Ottomata set the point value for this task to 3.
gerritbot added a comment.Aug 15 2018, 5:32 PM

Change 452958 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Use newer librdkafka for eventlogging

https://gerrit.wikimedia.org/r/452958

gerritbot added a project: Patch-For-Review.Aug 15 2018, 5:32 PM
Ottomata added a comment.Aug 15 2018, 5:34 PM

Currently running librdkafka1 0.11.5-1~bpo9+1 in deployment-prep. Will leave it for a day or two and then upgrade in prod.

Ottomata added a project: Analytics-Kanban.Aug 15 2018, 5:34 PM
Ottomata moved this task from Next Up to In Progress on the Analytics-Kanban board.
Ottomata moved this task from In Progress to Ready to Deploy on the Analytics-Kanban board.Aug 20 2018, 3:02 PM
gerritbot added a comment.Aug 20 2018, 6:09 PM

Change 452958 merged by Ottomata:
[operations/puppet@production] Use newer librdkafka for eventlogging

https://gerrit.wikimedia.org/r/452958

gerritbot added a comment.Aug 20 2018, 6:12 PM

Change 454074 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Fix librdkafka backports pin for eventlogging

https://gerrit.wikimedia.org/r/454074

gerritbot added a comment.Aug 20 2018, 6:13 PM

Change 454074 merged by Ottomata:
[operations/puppet@production] Fix librdkafka backports pin for eventlogging

https://gerrit.wikimedia.org/r/454074

Stashbot added a comment.Aug 20 2018, 6:18 PM

Mentioned in SAL (#wikimedia-analytics) [2018-08-20T18:18:34Z] <ottomata> restaring eventlogging client side processes using librdkafka1 0.11.x - https://phabricator.wikimedia.org/T200769

Ottomata moved this task from Ready to Deploy to Done on the Analytics-Kanban board.Aug 20 2018, 7:04 PM
• Nuria closed this task as Resolved.Aug 30 2018, 12:30 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits