Figure out the right configuration to send the uwsgi and django log streams from Striker on californium to Logstash.
Some things to look at:
This might also be a good time to make the logging configuration layer in Striker easier to override via Puppet generated files.
Change 305941 had a related patch set uploaded (by BryanDavis):
Add Logstash logging support
Change 306049 had a related patch set uploaded (by BryanDavis):
Add python-logstash and bump wheels
Change 306055 had a related patch set uploaded (by BryanDavis):
logstash: Tag Striker messages for indexing
Change 306081 had a related patch set uploaded (by BryanDavis):
logstash: new input for msgpack over UDP
The uwsgi side ended up being a bit uglier than I had hoped, but I got it to work on my testing cluster in Labs. The working uwsgi logging config looks something like this:
[uwsgi]
# Keep a copy of all stout/stderr logs locally
logger=local file:/srv/log/striker/main.log
log-route=local .*
log-encoder=format:local [${strftime:%%Y-%%m-%%dT%%H:%%M:%%S}] ${msgnl}
# Send a copy of all stdout/stderr logs to logstash as json datagrams
logger=logstash socket:deployment-logstash2.eqiad.wmflabs:11514
log-route=logstash .*
log-encoder=json:logstash {"@timestamp":"${strftime:%%Y-%%m-%%dT%%H:%%M:%%S}","type":"striker","logger_name":"uwsgi","host":"%h","level":"INFO","message":"${msg}"}
# Keep WMF-style Apache combined access logs locally
req-logger=file:/srv/log/striker/access.log
log-format=%(addr) - %(user) [%(ltime)] "%(method) %(uri) (proto)" %(status) %(size) "%(referer)" "%(uagent)" %(micros)I theory the log-route settings are not needed, but in practice I couldn't get everything to work as desired without them.