Page MenuHomePhabricator

wikimedia-event-utilities should provide tools for JVM based apps producing directly to kafka
Closed, ResolvedPublic8 Estimated Story Points

Description

As a user of the event platform I want wikimedia-event-utilities to have all the functionalities that event-gate provides so that I can use my own kafka producers to ship sane&well-formed events.

In the context of a flink application it is generally simpler to use existing kafka producer, but there needs to be tools available to make sure the json events that are produced are valid regarding the event-platform guidelines and wikimedia-event-utilities seems to be the best place to put these tools for JVM based applications.

AC:

  • there should be a way to validate an event against its json schema
  • there should utilities to help prefixing the kafka topic with the DC name
  • stream config should be used to restrict what types of events are allowed in which streams
  • anything that would help to prevent shipping invalid events

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Change 649896 had a related patch set uploaded (by DCausse; owner: DCausse):
[wikimedia-event-utilities@master] Add an utility class to validate an event against its schema

https://gerrit.wikimedia.org/r/649896

Change 649896 merged by Ottomata:
[wikimedia-event-utilities@master] Add an utility class to validate an event against its schema

https://gerrit.wikimedia.org/r/649896

CBogen set the point value for this task to 8.Jan 4 2021, 4:53 PM
CBogen moved this task from Incoming to In Progress on the Discovery-Search (Current work) board.

Change 654658 had a related patch set uploaded (by DCausse; owner: DCausse):
[wikimedia-event-utilities@master] Add JsonEventGenerator

https://gerrit.wikimedia.org/r/654658

Change 654658 merged by jenkins-bot:
[wikimedia-event-utilities@master] Add JsonEventGenerator

https://gerrit.wikimedia.org/r/654658