Page MenuHomePhabricator

Make varnishkafka produce using dynamic topics
Closed, DeclinedPublic

Description

As more varnish cache types are consolidated, more webrequest data is being sent to fewer Kafka topics. This is causing webrequest_source='text' partition in Hive/Hadoop to get huge.

We need varnishkafka to be smarter about the topics it produces to.