Page MenuHomePhabricator

Select a standard log shipping solution to use with applications that cannot be configured to send log events directly to Logstash and/or fluorine
Open, MediumPublic

Description

Ideally we would find a way to configure the application to ship logs directly to logstash and fluorine. Today we are shipping logs to fluorine via both our proprietary udp2log packet format and syslog. Logstash is actively accepting log events as GELF packets and syslog UDP datagrams. There are a number of additional log input methods that can be enabled for Logstash if needed.
If for some reason direct shipping is not possible, we should find a log shipper application that can tail the local disk logs and relay the events they represent to other hosts. If this is needed I'd like to work with someone from ops to find a single application that we can standardize on across production rather than having a one-off solution for each and every application that has an inflexible logging layer. There are a large number of solutions in this space including our own proprietary log2udp service, rsyslog's imfile feature, and dedicated log shipping services like beaver and lumberjack. I'd lean towards finding a dedicated shipper with an active community to standardize on as these tools generally provide a more flexible means to describe what constitutes a single log event than log2udp and rsyslog which are both strictly line oriented.

Event Timeline

bd808 created this task.Apr 27 2015, 3:47 PM
bd808 raised the priority of this task from to Needs Triage.
bd808 updated the task description. (Show Details)
bd808 added a subscriber: bd808.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 27 2015, 3:47 PM
bd808 added a comment.Apr 27 2015, 4:39 PM

The only things I have personally used before to do this are the forwarder bundled with Splunk (proprietary) and local installs of Logstash (a bit RAM/CPU heavy due to use of JVM).

The forwarder formerly known as Lumberjack seems promising if only because it is maintained by Elastic and they also maintain Logstash. A possible negative however is that it is written in Go which brings a new language to the cluster if we find that we need to write a custom extension for some reason. On initial examination it looks like it would be non-trivial to extend it to also send log events to udp2log on fluorine.

Dzahn triaged this task as Medium priority.Apr 29 2015, 1:44 AM
Dzahn added a subscriber: Dzahn.
Restricted Application added a subscriber: Matanya. · View Herald TranscriptAug 3 2015, 10:26 PM
fgiunchedi moved this task from Backlog to Up next on the Wikimedia-Logstash board.Aug 6 2018, 1:13 PM
fgiunchedi moved this task from Inbox to Up next on the observability board.Aug 19 2019, 2:57 PM
fgiunchedi moved this task from Up next to Inbox on the observability board.Nov 25 2019, 1:51 PM