Logging directly to logstash via the syslog input is more robust and matches current MediaWiki practice. Scap is one of 3 remaining udp2log input channels.
Description
Description
Details
Details
Related Changes in Gerrit:
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | herron | T227080 Deprecate all non-Kafka logstash inputs | |||
| Open | None | T205856 Retire udp2log: onboard its producers and consumers to the logging pipeline | |||
| Resolved | fgiunchedi | T86969 Send scap log directly to logstash via syslog input |
Event Timeline
Comment Actions
Scap already formats its messages for logstash so the change needed is to mimic the Monolog RedisHandler and rpush the log events on to the "logstash" list in redis on any of the logstash servers. The prod MW config randomly selects a redis host for each web request from the 3 possible. In beta there is only a single host to choose.
Config should allow choosing udp2log and/or redis as output and getting a list of redis servers based on the existing cascading config system. When redis is configured, the list of servers should be shuffled and the first or last one picked.
Comment Actions
The notes about redis input are horribly out of date. The redis input queue was killed fairly soon after being deployed.
Syslog is the transport used by MediaWiki these days, but there are several others possible. There are two raw json inputs: json codec over UDP on port 11514, and json_lines codec over TCP on port 11514 might be the easiest to use with a Python app like Scap. The python-logstash library or custom code should be pretty easy to use with either.
Comment Actions
Update on udp2log deprecation: we've deployed the new logging infrastructure now, IOW the recommended way is to use the system's syslog daemon (unix socket on /dev/log that is) and opt-in programs to have their syslog ingested into logstash via kafka. For json logging the syslog payload should be prefixed with @cee: to signal structured/json logging. I'm happy to provide further guidance too!
Comment Actions
Change 493232 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] deployment_server: ship logs through logging pipeline
Comment Actions
Change 493232 merged by Filippo Giunchedi:
[operations/puppet@production] deployment_server: ship logs through logging pipeline
Comment Actions
Change 563468 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[mediawiki/tools/scap@master] Support for logging json to syslog
Comment Actions
Change 563468 merged by jenkins-bot:
[mediawiki/tools/scap@master] Support for logging json to syslog
Comment Actions
Patch has been deployed, however I overlooked adding @cee and will followup with a fix
Comment Actions
Change 629342 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[mediawiki/tools/scap@master] log: add @cee: token for structured logging
Comment Actions
Change 629342 merged by jenkins-bot:
[mediawiki/tools/scap@master] log: add @cee: token for structured logging
Comment Actions
@LarsWirzenius re: the next scap release to get this change included, is there a timeline ? thanks!