Research sending EventLogging validation logs to Logstash {oryx} [5 pts]
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	mforns
	Sep 3 2015, 5:29 PM

Description

Implement a system to send EventLogging validation logs into Logstash to enable easy troubleshooting for non-analytics users.

Problem:
When developing new schemas for EventLogging, non-analytics schema owners have difficulties in checking if the events generated by their new instrumentation are indeed correct, or have validation problems. That is so, because they don't have access to eventlog1001.eqiad.wmnet, where the validation logs live (for security reasons). This results in unnecessary email ping-pong between the schema owners (who can not see the logs) and analytics developers (who can not modify the instrumentation code).

Solution attempt:
Modify the EventLogging code to additionally send the validation logs into Logstash in real time. Thus, the non-analytics developers will be able to instantly debug their instrumentation by accessing Logstash. Both their team and analytics will save considerable time.

research what is the best option:

consuming the kafka topic directly from logstash
sending events to logstash from a new EL writer

Testing on analytics1004.eqiad.wmnet

Related Objects

Mentioned Here: T113627: Make Logstash consume from Kafka:eventlogging_EventError {Oryx} [8 pts]

Event Timeline

mforns created this task.Sep 3 2015, 5:29 PM

mforns raised the priority of this task from to High.

mforns updated the task description. (Show Details)

mforns added projects: Analytics-Backlog, MediaWiki-extensions-EventLogging.

mforns subscribed.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 3 2015, 5:29 PM

mforns moved this task from Incoming to Prioritized on the Analytics-Backlog board.Sep 3 2015, 5:29 PM

With {stag} (and currently but not official yet), all event processing errors go to a special topic in Kafka 'eventlogging_EventError'. EventError is itself an eventlogging schema, which has a field that contains the original raw event string that failed processing, and if possible, a descriptive error message as to why it failed.

This topic can be consumed from Kafka via eventlogging or other kafka tools.

This is awesome! \o/
Probably there's a tool to easily move those to Logstash,
Will look for it.
Thx

• kevinator renamed this task from Send EventLogging validation logs to Logstash {oryx?} to Send EventLogging validation logs to Logstash {oryx}.Sep 14 2015, 4:35 PM

• kevinator set Security to None.

• kevinator renamed this task from Send EventLogging validation logs to Logstash {oryx} to Research sending EventLogging validation logs to Logstash {oryx} [5 pts] .Sep 14 2015, 4:48 PM

• kevinator updated the task description. (Show Details)

https://www.elastic.co/guide/en/logstash/current/plugins-inputs-kafka.html

mforns claimed this task.Sep 21 2015, 10:20 PM

mforns edited projects, added Analytics-Kanban; removed Analytics-Backlog.

mforns moved this task from Next Up to In Progress on the Analytics-Kanban board.

Milimetric moved this task from In Progress to Done on the Analytics-Kanban board.Sep 24 2015, 3:45 PM

• kevinator closed this task as Resolved.Sep 25 2015, 3:38 PM

• kevinator subscribed.

Is this done? I missed standup! I want to see! :)

The research part is done.

WMF's Logstash has support for native Kafka consumer plugin. So, we'll go for it in the next task, the implementation one: T113627. I've already emailed with Bryan Davis and he seems optimistic about it.

Research sending EventLogging validation logs to Logstash {oryx} [5 pts] Closed, ResolvedPublicActions

Description

Related Objects

Event Timeline

Research sending EventLogging validation logs to Logstash {oryx} [5 pts]
Closed, ResolvedPublic
Actions