Page MenuHomePhabricator

Logstash syslog input grok parse failure on some network devices lines
Closed, ResolvedPublic

Description

While investigating T213899: Migrate at least 3 existing Logstash inputs and associated producers to the new Kafka-logging pipeline, and remove the associated non-Kafka Logstash inputs I ran into some syslog lines that had a _grokparsefailure_sysloginput tag, e.g.:

{
  "_index": "logstash-syslog-2019.02.25",
  "_type": "syslog",
  "_id": "AWkk2uwsZ9NltGfJqQ1p",
  "_version": 1,
  "_score": null,
  "_source": {
    "severity": 0,
    "level": "EMERGENCY",
    "message": "<158>Feb 25 13:32:00 /usr/sbin/cron[86116]: (root) CMD (newsyslog)",
    "type": "syslog",
    "priority": 0,
    "normalized_message": "<158>Feb 25 13:32:00 /usr/sbin/cron[86116]: (root) CMD (newsyslog)",
    "tags": [
      "input-syslog-10514",
      "_grokparsefailure_sysloginput",
      "syslog",
      "es",
      "normalized_message_untrimmed"
    ],
    "@timestamp": "2019-02-25T13:32:00.716Z",
    "@version": "1",
    "host": "%{logsource}",
    "facility": 0,
    "severity_label": "Emergency",
    "facility_label": "kernel"
  },
  "fields": {
    "@timestamp": [
      1551101520716
    ]
  },
  "sort": [
    1551101520716
  ]
}

And on the wire I think these messages look something like this:

<158>Feb 25 13:45:00 asw2-a5-eqiad /usr/sbin/cron[12760]: %-: (root) CMD (newsyslog)

Event Timeline

As of the last 90 days, something does periodically try to send malformed input to the deprecated syslog input, but there does not appear to be any valid traffic being handled by this input. It seems safe to remove.

query: tags:input-syslog-10514 AND NOT tags:_grokparsefailure_sysloginput

Change 662009 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] profile: remove deprecated syslog input

https://gerrit.wikimedia.org/r/662009

Change 663214 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] hiera: prepare logstash syslog lvs config for removal

https://gerrit.wikimedia.org/r/663214

Change 663214 merged by Cwhite:
[operations/puppet@production] hiera: prepare logstash syslog lvs config for removal

https://gerrit.wikimedia.org/r/663214

Change 663242 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] hiera: prepare logstash-syslog lvs service for removal

https://gerrit.wikimedia.org/r/663242

Change 663242 merged by Cwhite:
[operations/puppet@production] hiera: prepare logstash-syslog lvs service for removal

https://gerrit.wikimedia.org/r/663242

Change 662009 merged by Cwhite:
[operations/puppet@production] profile: remove deprecated syslog input

https://gerrit.wikimedia.org/r/662009

logstash-syslog removed from production