Page MenuHomePhabricator

haproxykafka: feature: Ability to print structured messages to stdout
Open, Needs TriagePublic

Description

A nice feature to have is the ability to start printing structured messages (json) to stdout/file pipe/socket while keep sending all data to the Kafka cluster. This is extremely useful for live debugging, assuming there's a handy way to start/stop this behavior at runtime (eg. sending a specific signal to the process).

As commented by @Vgutierrez we should be careful about eventual PII that could be written on disk (or journal in case of stdout) and about performances too (one of the biggest issues with Benthos regarding this).

Event Timeline

This would need some filtering and we should be very careful on how it's implemented to not trigger a big performance degradation. I'd say that stdout/file is ruled out as we don't want PII hitting the disk (I include stdout here cause systemd persists it to disk)

This would need some filtering and we should be very careful on how it's implemented to not trigger a big performance degradation. I'd say that stdout/file is ruled out as we don't want PII hitting the disk (I include stdout here cause systemd persists it to disk)

Thanks, description updated