Page MenuHomePhabricator

Investigate if kafka can decline requests to consume from consumers that support an older protocol
Open, NormalPublic

Description

Seems that since we control most kafka consumers we should not allow consumers to consume with " too old protocols " if possible. Seems that the translations among protocols is costly for kafka and part of the cause of the OOMs we were seeing.

Event Timeline

Nuria created this task.Apr 2 2019, 9:36 PM
Nuria added a comment.Apr 2 2019, 11:31 PM

Of interest: https://github.com/apache/kafka/blob/0.10.0/core/src/main/scala/kafka/server/KafkaApis.scala#L436

I think the magic byte is what triggers the conversion among api versions

fdans triaged this task as High priority.Apr 4 2019, 5:06 PM
fdans lowered the priority of this task from High to Normal.
fdans moved this task from Incoming to Operational Excellence on the Analytics board.

Until we get rid of Camus, we can't restrict old client versions. Camus uses a 0.8 client. :/