HomePhabricator

logstash: Update logstash for sending to es 2.x
17db941f1ad9Unpublished

Unpublished Commit · Learn More

Not On Permanent Ref: This commit is not an ancestor of any permanent ref.
This commit no longer exists in the repository. It may have been part of a branch which was deleted.This commit has been deleted in the repository: it is no longer reachable from any branch, tag, or ref.

Description

logstash: Update logstash for sending to es 2.x

Adjust index mapping template to be 2.x compatible:

  • Remove path: full from geoip. This is now unsupported. full was already the default, and is what new versions of es always do so no other changes required.
  • Remove 'index_name: tag' from tags. This was aliasing the tag field to tags. It is no longer supported in 2.x. Any dashboards that query tag must query tags instead now.

Post process logs to make them 2.x compatible:

  • converts dot's in the properties into underscores. 2.x does not allow dots in properties, they are used as separators.
  • Normalizes the pid and line fields into int. Some of the log types for nodejs services auto-created these as strings.
  • Drops the extra timestamp field. @timestamp is definitive, and this timestamp field is seen sometimes as a date, sometimes as a string in the auto generated mappings depending on the content of the field the first time it was seen.

Change-Id: I46d177ce1218eee6f86fa9468b917dc54b3d55da
Bug: T138335

Details

Provenance
EBernhardsonAuthored on Jun 22 2016, 7:48 PM
ChangeId
I46d177ce1218eee6f86fa9468b917dc54b3d55da

Commit No Longer Exists

This commit no longer exists in the repository.