During this incident, 2025-03-12 ExternalStorage Database Cluster Overload, MediaWiki emitted millions of events to channel:Memcached, for thousands of requests.
In this incident, mw-{api-int,parsoid, jobrunner} were trying to connect to an invalid memcached server address, no to avail. Thus, they sent out millions of messages per 5 mins.
During this particular incident, this behaviour overwhelmed logstash, causing delays and alerts not being fired.
Below we can see envoy's POV. While I have not done the math yet, but my imprerssion is that we are logging quite aggressively.
