Update to node-rdkafka 0.8.0
Closed, ResolvedPublic

Description

A new release of the node Kafka driver has been cut. It brings some important fixes and has the potential to fix the memory leak we've been experiencing with ChangeProp (T153215)

This version depends on a newer version of librdkafka, so we need to make a coordinated update with all the services. There's some backwards incompatible API changes, so services using the driver should be updated:

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 28 2017, 12:24 AM

Change 340264 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
Update node-rdkafka to 0.8.0

https://gerrit.wikimedia.org/r/340264

Change 341597 had a related patch set uploaded (by Ottomata):
[mediawiki/services/eventstreams] Update to kafkasse 0.1.0 which depends on node-rdkafka 0.8.0

https://gerrit.wikimedia.org/r/341597

Change 341597 merged by Ottomata:
[mediawiki/services/eventstreams] Update to kafkasse 0.1.0 which depends on node-rdkafka 0.8.0

https://gerrit.wikimedia.org/r/341597

Change 340264 merged by Mobrovac:
[mediawiki/services/trending-edits] Update node-rdkafka to 0.8.0

https://gerrit.wikimedia.org/r/340264

Change 342850 had a related patch set uploaded (by Ottomata):
[mediawiki/services/eventstreams] Pull librdkafka from apt.wikimedia.org jessie-wikimedia backports

https://gerrit.wikimedia.org/r/342850

Change 342850 merged by Ottomata:
[mediawiki/services/eventstreams] Pull librdkafka from apt.wikimedia.org jessie-wikimedia backports

https://gerrit.wikimedia.org/r/342850

mobrovac edited the task description. (Show Details)Mar 15 2017, 4:40 PM

Mentioned in SAL (#wikimedia-operations) [2017-03-15T17:41:52Z] <ppchelko@tin> Started deploy [trending-edits/deploy@85be190]: Trending: Update to node-rdkafka 0.8.0. Canary on scb2001. T159200

Mentioned in SAL (#wikimedia-operations) [2017-03-15T17:42:49Z] <mobrovac@tin> Started deploy [changeprop/deploy@614cb4b]: Canary deploy for switching to librdkafka 0.9.4 T159200

Mentioned in SAL (#wikimedia-operations) [2017-03-15T17:43:13Z] <ppchelko@tin> Finished deploy [trending-edits/deploy@85be190]: Trending: Update to node-rdkafka 0.8.0. Canary on scb2001. T159200 (duration: 01m 21s)

Mentioned in SAL (#wikimedia-operations) [2017-03-15T17:43:42Z] <mobrovac@tin> Finished deploy [changeprop/deploy@614cb4b]: Canary deploy for switching to librdkafka 0.9.4 T159200 (duration: 00m 53s)

Mentioned in SAL (#wikimedia-operations) [2017-03-15T17:46:40Z] <otto@tin> Started deploy [eventstreams/deploy@eb8698e]: T159200

Mentioned in SAL (#wikimedia-operations) [2017-03-15T17:46:58Z] <otto@tin> Finished deploy [eventstreams/deploy@eb8698e]: T159200 (duration: 00m 17s)

Mentioned in SAL (#wikimedia-operations) [2017-03-15T17:50:08Z] <ottomata> upgrading librdkafka on scb in codfw T159200

Mentioned in SAL (#wikimedia-operations) [2017-03-15T17:50:55Z] <otto@tin> Started deploy [eventstreams/deploy@eb8698e]: T159200

Mentioned in SAL (#wikimedia-operations) [2017-03-15T17:51:08Z] <ppchelko@tin> Started deploy [trending-edits/deploy@85be190]: Update to node-rdkafka 0.8.0 in codfw. T159200

Mentioned in SAL (#wikimedia-operations) [2017-03-15T17:51:31Z] <mobrovac@tin> Started deploy [changeprop/deploy@614cb4b]: Deploy to CODFW for switching to librdkafka 0.9.4 T159200

Mentioned in SAL (#wikimedia-operations) [2017-03-15T17:52:30Z] <otto@tin> Finished deploy [eventstreams/deploy@eb8698e]: T159200 (duration: 01m 35s)

Mentioned in SAL (#wikimedia-operations) [2017-03-15T17:53:15Z] <mobrovac@tin> Finished deploy [changeprop/deploy@614cb4b]: Deploy to CODFW for switching to librdkafka 0.9.4 T159200 (duration: 01m 44s)

Mentioned in SAL (#wikimedia-operations) [2017-03-15T17:55:00Z] <ppchelko@tin> Finished deploy [trending-edits/deploy@85be190]: Update to node-rdkafka 0.8.0 in codfw. T159200 (duration: 03m 51s)

Mentioned in SAL (#wikimedia-operations) [2017-03-15T18:06:15Z] <otto@tin> Started deploy [eventstreams/deploy@eb8698e]: T159200

Mentioned in SAL (#wikimedia-operations) [2017-03-15T18:06:31Z] <ppchelko@tin> Started deploy [trending-edits/deploy@85be190]: Update to node-rdkafka 0.8.0. Canary on scb1001.eqiad.wmnet. T159200

Mentioned in SAL (#wikimedia-operations) [2017-03-15T18:07:38Z] <ppchelko@tin> Finished deploy [trending-edits/deploy@85be190]: Update to node-rdkafka 0.8.0. Canary on scb1001.eqiad.wmnet. T159200 (duration: 01m 07s)

Mentioned in SAL (#wikimedia-operations) [2017-03-15T18:07:43Z] <mobrovac@tin> Started deploy [changeprop/deploy@614cb4b]: Deploy to EQIAD canary for switching to librdkafka 0.9.4 T159200

Mentioned in SAL (#wikimedia-operations) [2017-03-15T18:07:59Z] <mobrovac@tin> Finished deploy [changeprop/deploy@614cb4b]: Deploy to EQIAD canary for switching to librdkafka 0.9.4 T159200 (duration: 00m 20s)

Mentioned in SAL (#wikimedia-operations) [2017-03-15T18:12:52Z] <ottomata> upgrading librdkafka on scb eqiad nodes T159200

Mentioned in SAL (#wikimedia-operations) [2017-03-15T18:13:43Z] <otto@tin> Started deploy [eventstreams/deploy@eb8698e]: T159200

Mentioned in SAL (#wikimedia-operations) [2017-03-15T18:13:48Z] <ppchelko@tin> Started deploy [trending-edits/deploy@85be190]: Update to node-rdkafka 0.8.0. T159200

Mentioned in SAL (#wikimedia-operations) [2017-03-15T18:15:07Z] <mobrovac@tin> Started deploy [changeprop/deploy@614cb4b]: Deploy for switching to librdkafka 0.9.4 T159200

Mentioned in SAL (#wikimedia-operations) [2017-03-15T18:15:40Z] <mobrovac@tin> Finished deploy [changeprop/deploy@614cb4b]: Deploy for switching to librdkafka 0.9.4 T159200 (duration: 00m 33s)

Mentioned in SAL (#wikimedia-operations) [2017-03-15T18:19:58Z] <ppchelko@tin> Finished deploy [trending-edits/deploy@85be190]: Update to node-rdkafka 0.8.0. T159200 (duration: 06m 11s)

Mentioned in SAL (#wikimedia-operations) [2017-03-15T18:20:03Z] <otto@tin> Finished deploy [eventstreams/deploy@eb8698e]: T159200 (duration: 06m 18s)

Mentioned in SAL (#wikimedia-operations) [2017-03-15T19:15:30Z] <mobrovac@tin> Started deploy [changeprop/deploy@b68bf51]: Deploy producer fix for T159200

Mentioned in SAL (#wikimedia-operations) [2017-03-15T19:16:21Z] <mobrovac@tin> Finished deploy [changeprop/deploy@b68bf51]: Deploy producer fix for T159200 (duration: 00m 51s)

mobrovac closed this task as "Resolved".
mobrovac triaged this task as "Normal" priority.
mobrovac removed a project: Patch-For-Review.
mobrovac edited the task description. (Show Details)
mobrovac edited the task description. (Show Details)
mobrovac removed a subscriber: gerritbot.