Page MenuHomePhabricator

Increase Benthos capacity
Open, Needs TriagePublic

Description

In situations where there are significant spikes in log traffic at the edge via haproxykafka, benthos can become overloaded and there is delay in webrequest data. Currently we have one benthos instance per centrallog host, which is a risk factor and also not enough capacity for these spikes.

We should investigate adding benthos capacity - Kubernetes seems like a natural fit for this purpose as we already have benthos charts in use that will more or less perfectly fit the approach that we currently use on the centrallog hosts.