Maniphest T138335

Backport de_dot plugin for logstash to 1.5.3
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	EBernhardson
	Jun 21 2016, 5:21 PM

Tags

Referenced Files

None

Subscribers

Description

Elasticsearch 2.x no longer allows dot's in property names. Logstash has since added a de_dot[1] plugin to help users deal with this. We do not appear to have any need to do a full test/upgrade cycle on logstash as part of the es2.x upgrade, instead we should backport this plugin to the version of logstash we currently use.

Details

	Subject	Repo	Branch	Lines +/-
	logstash: Update logstash for sending to es 2.x	operations/puppet	production	+25 -2
	Add de_dot filter and rename to logstash-filters-wikimedia	operations/software/logstash/plugins	master	+99 -2

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		bd808	T137400 Logstash elasticsearch mapping does not allow err.code to be a string
		Resolved		dcausse	T138608 Vagrant role provisions wrong elasticsearch version
		Resolved		debt	T136001 [EPIC] Upgrade elasticsearch cluster supporting logging to 2.3
		Resolved		EBernhardson	T136003 Upgrade logstash*.eqiad.wmnet elasticsearch servers to 2.3
		Resolved		EBernhardson	T138335 Backport de_dot plugin for logstash to 1.5.3

Event Timeline

EBernhardson created this task.Jun 21 2016, 5:21 PM

Restricted Application added subscribers: Zppix, Aklapper. · View Herald TranscriptJun 21 2016, 5:21 PM

EBernhardson added a parent task: T136003: Upgrade logstash*.eqiad.wmnet elasticsearch servers to 2.3.Jun 21 2016, 5:21 PM

EBernhardson moved this task from needs triage to Up Next on the Discovery-Search board.Jun 21 2016, 10:09 PM

EBernhardson edited projects, added Discovery-Search (Current work); removed Discovery-Search.

EBernhardson claimed this task.Jun 22 2016, 4:52 PM

EBernhardson moved this task from Incoming to not in use - please delete on the Discovery-Search (Current work) board.

EBernhardson moved this task from not in use - please delete to Needs review on the Discovery-Search (Current work) board.Jun 22 2016, 7:42 PM

Change 295575 had a related patch set uploaded (by EBernhardson):
Add de_dot filter and rename to logstash-filters-wikimedia

https://gerrit.wikimedia.org/r/295575

gerritbot added a project: Patch-For-Review.Jun 22 2016, 7:42 PM

Change 295578 had a related patch set uploaded (by EBernhardson):
logstash: Utilize de_dot filter for sending to es 2.x

https://gerrit.wikimedia.org/r/295578

EBernhardson mentioned this in rOPUPe549bf58fcea: logstash: Utilize de_dot filter for sending to es 2.x.Jun 22 2016, 7:56 PM

EBernhardson mentioned this in rOPUP6b72f5f85fab: logstash: Utilize de_dot filter for sending to es 2.x.

EBernhardson mentioned this in rOPUP828912826a4d: logstash: Utilize de_dot filter for sending to es 2.x.Jun 22 2016, 9:14 PM

EBernhardson mentioned this in rOPUP1d07fdcf170b: logstash: Update filters for sending to es 2.x.Jun 23 2016, 4:51 PM

EBernhardson mentioned this in rOPUP35074756b801: logstash: Update filters for sending to es 2.x.Jun 23 2016, 4:59 PM

EBernhardson mentioned this in rOPUP3f256793cb83: logstash: Update filters for sending to es 2.x.Jun 23 2016, 6:13 PM

EBernhardson mentioned this in rOPUP7b7b129c0cfb: logstash: Update filters for sending to es 2.x.Jun 23 2016, 6:17 PM

EBernhardson mentioned this in rOPUPf243f99c9be7: logstash: Update filters for sending to es 2.x.Jun 23 2016, 6:21 PM

EBernhardson mentioned this in rOPUPb00ec8697ec7: logstash: Update logstash for sending to es 2.x.Jun 23 2016, 6:30 PM

Change 295575 merged by BryanDavis:
Add de_dot filter and rename to logstash-filters-wikimedia

https://gerrit.wikimedia.org/r/295575

EBernhardson mentioned this in rOPUP8b1a1198fe0e: logstash: Update logstash for sending to es 2.x.Jun 23 2016, 8:47 PM

EBernhardson moved this task from Needs review to Needs Reporting on the Discovery-Search (Current work) board.Jun 23 2016, 9:34 PM

EBernhardson mentioned this in rOPUP17db941f1ad9: logstash: Update logstash for sending to es 2.x.Jun 27 2016, 2:48 PM

Gehel mentioned this in rOPUP99dd186c4189: logstash: Update logstash for sending to es 2.x.Jun 27 2016, 5:17 PM

Change 295578 merged by Gehel:
logstash: Update logstash for sending to es 2.x

https://gerrit.wikimedia.org/r/295578

Mentioned in SAL [2016-06-27T17:23:22Z] <gehel> deploying new logstash config for transition to elasticsearch 2.x (T138335)

Working in production since rOSLP18b3f1fe3d44: Fix de_dot to process keys with falsey values

fgiunchedi added a project: observability.Aug 19 2019, 2:27 PM