Page MenuHomePhabricator
Create Task
Maniphest T179786

Update trending-edits' node-rdkafka to v1.x
Closed, ResolvedPublic
Actions

Description

The Trending Edits service is still using node-rdkafka version 0.8.x. Since the other SCB services are using version 1.x, we should update this service to it ASAP.

Details

SubjectRepoBranchLines +/-
Update node-rdkafka dependency to 1.xmediawiki/services/trending-editsmaster+1 -1
Update node-rdkafka dependency to 2.xmediawiki/services/trending-editsmaster+1 -1
Customize query in gerrit

Related Objects

Event Timeline

• mobrovac created this task.Nov 5 2017, 10:03 AM
Restricted Application added a project: Product-Infrastructure-Team-Backlog-Deprecated. · View Herald TranscriptNov 5 2017, 10:03 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Jdlrobson added a project: User-Jdlrobson.Nov 6 2017, 4:39 PM
Jdlrobson moved this task from Inbox to Tracking on the User-Jdlrobson board.Nov 7 2017, 8:06 PM
• Fjalapeno edited projects, added Product-Infrastructure-Team-Backlog-Deprecated (Kanban); removed Product-Infrastructure-Team-Backlog-Deprecated.Nov 9 2017, 4:12 PM
• mobrovac added a project: Wikimedia-Incident.Nov 10 2017, 11:53 AM

We have seen now twice in one week trending-edits swallowing memory on SCB nodes because the service's offsets disappear from Kafka. This is likely due to the fact that this service is the only one using an older version of node-rdkafka which was not compiled for the version present on SCB. The dependency has to be updated ASAP. @Jdlrobson could you please take care of this?

Joe subscribed.Nov 10 2017, 11:59 AM

I would be tempted to raise the priority of this task to UBN since this has cost debug sessions twice in a week to multiple people. I'm also adding operations to the task as we're clearly impacted.

Joe added projects: SRE, User-Joe.Nov 10 2017, 11:59 AM
gerritbot subscribed.Nov 10 2017, 10:47 PM

Change 390544 had a related patch set uploaded (by BearND; owner: BearND):
[mediawiki/services/trending-edits@master] Update node-rdkafka dependency to 1.x

https://gerrit.wikimedia.org/r/390544

gerritbot added a project: Patch-For-Review.Nov 10 2017, 10:47 PM

Change 390546 had a related patch set uploaded (by BearND; owner: BearND):
[mediawiki/services/trending-edits@master] Update node-rdkafka dependency to 2.x

https://gerrit.wikimedia.org/r/390546

• bearND subscribed.Nov 10 2017, 10:50 PM

There's already a 2.2.0 out. Pushed an additional commit for that.

• mobrovac added a comment.Nov 11 2017, 10:47 AM

Thank you, @bearND for looking into it.

In T179786#3752001, @bearND wrote:

There's already a 2.2.0 out. Pushed an additional commit for that.

Heh, we are not ready for v2.x yet, cf. T176126: Update node-rdkafka version to v2.x

gerritbot added a comment.Nov 12 2017, 3:18 PM

Change 390546 abandoned by BearND:
Update node-rdkafka dependency to 2.x

Reason:
We're not ready for 2.x. See T176126.

https://gerrit.wikimedia.org/r/390546

gerritbot added a comment.Nov 13 2017, 3:27 PM

Change 390544 merged by jenkins-bot:
[mediawiki/services/trending-edits@master] Update node-rdkafka dependency to 1.x

https://gerrit.wikimedia.org/r/390544

elukey subscribed.Nov 13 2017, 5:10 PM
greg moved this task from Active investigation to Active Situation on the Wikimedia-Incident board.Nov 14 2017, 11:15 PM
Stashbot subscribed.Nov 15 2017, 9:52 AM

Mentioned in SAL (#wikimedia-operations) [2017-11-15T09:51:59Z] <ppchelko@tin> Started deploy [trending-edits/deploy@a0e1fe3]: Update node-rdkafka to v1.x T179786

Stashbot added a comment.Nov 15 2017, 9:52 AM

Mentioned in SAL (#wikimedia-operations) [2017-11-15T09:52:36Z] <ppchelko@tin> Started deploy [trending-edits/deploy@a0e1fe3]: Update node-rdkafka to v1.x T179786

Stashbot added a comment.Nov 15 2017, 9:55 AM

Mentioned in SAL (#wikimedia-operations) [2017-11-15T09:55:16Z] <ppchelko@tin> Finished deploy [trending-edits/deploy@a0e1fe3]: Update node-rdkafka to v1.x T179786 (duration: 02m 44s)

Stashbot added a comment.Nov 15 2017, 9:55 AM

Mentioned in SAL (#wikimedia-operations) [2017-11-15T09:55:44Z] <ppchelko@tin> Started deploy [trending-edits/deploy@a0e1fe3]: Update node-rdkafka to v1.x attempt 2 T179786

Stashbot added a comment.Nov 15 2017, 9:59 AM

Mentioned in SAL (#wikimedia-operations) [2017-11-15T09:59:16Z] <ppchelko@tin> Finished deploy [trending-edits/deploy@a0e1fe3]: Update node-rdkafka to v1.x attempt 2 T179786 (duration: 03m 32s)

Stashbot added a comment.Nov 15 2017, 9:59 AM

Mentioned in SAL (#wikimedia-operations) [2017-11-15T09:59:36Z] <ppchelko@tin> Started deploy [trending-edits/deploy@a0e1fe3]: Update node-rdkafka to v1.x attempt 3, force T179786

Stashbot added a comment.Nov 15 2017, 10:04 AM

Mentioned in SAL (#wikimedia-operations) [2017-11-15T10:04:21Z] <ppchelko@tin> Finished deploy [trending-edits/deploy@a0e1fe3]: Update node-rdkafka to v1.x attempt 3, force T179786 (duration: 04m 44s)

Stashbot added a comment.Nov 15 2017, 10:04 AM

Mentioned in SAL (#wikimedia-operations) [2017-11-15T10:04:36Z] <ppchelko@tin> Started deploy [trending-edits/deploy@a0e1fe3]: Update node-rdkafka to v1.x attempt 4 T179786

Stashbot added a comment.Nov 15 2017, 10:09 AM

Mentioned in SAL (#wikimedia-operations) [2017-11-15T10:09:27Z] <ppchelko@tin> Finished deploy [trending-edits/deploy@a0e1fe3]: Update node-rdkafka to v1.x attempt 4 T179786 (duration: 04m 51s)

• Pchelolo closed this task as Resolved.Nov 15 2017, 10:11 AM
• Pchelolo claimed this task.
• Pchelolo subscribed.

Deployed. Resolving.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits