The logs from the various mailman3 components should end up in logstash.
|Resolved||Security||None||T181803 Stop storing Mailman passwords in plain text|
|Resolved||None||T118641 Implement proper AAA for lists.wikimedia.org (mailman)|
|Resolved||None||T190054 List archives on lists.wikimedia.org is not mobile friendly|
|Resolved||None||T115329 "From" at start of line becomes ">From" in pipermail|
|Resolved||None||T52864 Upgrade GNU Mailman from 2.1 to Mailman3|
|Open||None||T276697 Implement central logging for mailman3|
Here's what the logs to disk look like:
root@mailman-mailman02:~# tree /var/log/mailman3/ /var/log/mailman3/ ├── bounce.log ├── debug.log ├── mailman.log ├── mailman.log.1 ├── mailman.log.2.gz ├── mailman.log.3.gz ├── mailman.log.4.gz ├── mailman.log.5.gz ├── plugins.log ├── smtp.log └── web ├── mailman-web.log ├── mailman-web.log.1 ├── mailman-web.log.2.gz ├── mailman-web.log.3.gz ├── mailman-web.log.4.gz └── mailman-web.log.5.gz
What's the preferred way to get these into logstash?
I remember writing something for ores that sends all log entries to logstash (it was also uwsgi so it should be really similar) if you want to look it up but I'm not 100% sure we should send all uwsgi logs to logstash (I think it's possible to send >= WARN level logs to logstash only while logging in a lower level in file.). I can take a stab at it.
For daemons that are logging to syslog/journald the tl;dr to get the logs in logstash is to add the "program name" to modules/profile/files/rsyslog/lookup_table_output.json with value kafka local (or only kafka if you are not interested in local logs). For daemons logging to local files, tl;dr similar setup plus the "input file" part of rsyslog (i.e. rsyslog::input::file). Hope that helps! Happy to review patches of course and/or provide more guidance