Page MenuHomePhabricator

create a sampled log of POST data
Open, MediumPublic

Description

during a recent incident it was observed that it would be nice to have a sampling of the POST body contents logged somewhere, at least the first N bytes 1/1K times in short-term somewhere

Event Timeline

Dzahn added a parent task: Restricted Task.May 25 2022, 4:49 PM

We have something like this for POST requests to api.php on appservers, which we log (unsampled) to api.log on the mwlog host (excluded from Logstash). I've updated the documentation about this and other log files at https://wikitech.wikimedia.org/wiki/Logs.

I'm not suggesting that we take the same approach per-se. But, it's an option. Though regardless of approach, we'll probably want to sample these and/or limit to a particular appserver host. If we approach it from within MW (e.g. WikimediaEvents extension), we'd do good to do it from a post-send deferred update so as to take away any runtime overhead.

We also have something similar for GET requests for any kind, with (sampled, from small set of apaches), which we send to apache2.log and to Logstash.