The type, channel, level, and normalized_message fields are not in the ECS schema and unsuitable for throttling the ECS pipeline. Current behavior will throttle the whole ECS pipeline uniformly as the key will render the same for every ECS-formatted event.
Description
Description
Status | Subtype | Assigned | Task | ||
---|---|---|---|---|---|
Resolved | colewhite | T277813 Logstash is throttling the whole ECS pipeline | |||
Open | None | T277816 Improve Logstash's rate-limiting capabilities | |||
Resolved | EBernhardson | T290913 High rate of deprecation warning produced by elastic* nodes. | |||
Open | None | T295939 Logstash throttler does not apply to k8s logs | |||
Resolved | colewhite | T313099 Increase of ~50 million access logs per day from mobileapps-production-tls-proxy |
Event Timeline
Comment Actions
Resolved with the rollout of https://gerrit.wikimedia.org/r/c/operations/puppet/+/633224