Page MenuHomePhabricator

Processor writes valid and invalid events to separate Kafka topics {stag} [13 points]
Closed, ResolvedPublic

Description

for Python & Kafka on Beta Labs

Dan to pair with Marcel

Event Timeline

ggellerman assigned this task to Milimetric.
ggellerman raised the priority of this task from to Needs Triage.
ggellerman updated the task description. (Show Details)
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 11 2015, 9:19 PM
Milimetric renamed this task from Code to change the Processor to write invalid events to a separate Kafka topic from valid events [13 points] to Processor writes valid and invalid events to separate Kafka topics [13 points] {oryx}.May 12 2015, 3:46 PM
Milimetric set Security to None.
Milimetric added a project: Analytics-Kanban.
Milimetric moved this task from Next Up to In Progress on the Analytics-Kanban board.

Progress so far - I have a code change that should work but I don't know how to use kafka on beta labs. Zookeeper seems to not be running there.

Change 210729 had a related patch set uploaded (by Milimetric):
Add optional output for invalid events

https://gerrit.wikimedia.org/r/210729

Change 210765 had a related patch set uploaded (by Milimetric):
[WIP] Add parallel kafka pipeline

https://gerrit.wikimedia.org/r/210765

Change 210729 merged by Ottomata:
Add optional output for invalid events

https://gerrit.wikimedia.org/r/210729

ggellerman reassigned this task from Milimetric to Ottomata.Jun 8 2015, 3:34 PM
ggellerman moved this task from Paused to In Progress on the Analytics-Kanban board.
ggellerman added a subscriber: Milimetric.
kevinator renamed this task from Processor writes valid and invalid events to separate Kafka topics [13 points] {oryx} to Processor writes valid and invalid events to separate Kafka topics [13 points] {stag}.Jun 8 2015, 10:24 PM

Change 210765 merged by Ottomata:
Modify eventlogging module to use new changes to eventlogging server

https://gerrit.wikimedia.org/r/210765

kevinator renamed this task from Processor writes valid and invalid events to separate Kafka topics [13 points] {stag} to Processor writes valid and invalid events to separate Kafka topics {stag} [13 points].Jun 24 2015, 3:40 PM
kevinator triaged this task as Normal priority.Jul 2 2015, 3:37 PM
ggellerman moved this task from Ready to Deploy to Done on the Analytics-Kanban board.

work for this task is done and deployed... but code not being used right now. Marking as done as we have moved on.

kevinator closed this task as Resolved.Aug 4 2015, 4:01 PM