Change Details

As part of Modern Event Platform, we plan to build a scalable event stream intake system that will allow us to accept validated events from both internal and external clients. The working name of this component will be Stream Intake Service (subject to change). This system needs to be reliable and horizontally scalable. It should accept events synchronously (with validation responses, etc.) and asynchronously. It should accept events via HTTP POST. This component also includes any client libraries we might build that interact with the Event Intake Service as part of this component. See also {T225237} # User Stories [draft] - As an **engineer**, I want to produce valid events from internal networks so production features can use event systems. - As an **engineer**, I want to produce valid events from external clients so we can do comprehensive analysis based on events. - As an **engineer**, I want to produce events and get an appropriate response if my event is produced so I can build production backend features. - As an **engineer**, I want to produce events without a response if my event is produced so I collect "fire and forget" analytics data. - As an **engineer**, I want to batch produce many events at once so mobile apps can produce events after an offline period. - As an **engineer**, I want good client libraries to produce events so that I don't have to write them myself. - As an **engineer**, I want client libraries that can query for sampling settings and other schema topic usage metadata. - As an **engineer/analyst**, I want to be able to trigger events in a developer mode, regardless of sampling settings, so I can more easily debug problems. - As an **engineer**, I want to restrict usages of my event topic via authentication and authorization so that I don't get garbage data from internet trolls. - As an **analyst**, I want alarms for implausible value combinations (if field A has value X, then field B has to be > 200, or such) (this might not be part of event intake, more likely it is stream processing)

As part of Modern Event Platform, we plan to build a scalable event stream intake system that will allow us to accept validated events from both internal and external clients. The working name of this component will be Stream Intake Service (subject to change). This system needs to be reliable and horizontally scalable. It should accept events synchronously (with validation responses, etc.) and asynchronously. It should accept events via HTTP POST. This component also includes any client libraries we might build that interact with the Event Intake Service as part of this component. See also {T225237} # User Stories [draft] [x] As an **engineer**, I want to produce valid events from internal networks so production features can use event systems. [x] As an **engineer**, I want to produce valid events from external clients so we can do comprehensive analysis based on events. [x] As an **engineer**, I want to produce events and get an appropriate response if my event is produced so I can build production backend features. [x] As an **engineer**, I want to produce events without a response if my event is produced so I collect "fire and forget" analytics data. [x] As an **engineer**, I want to batch produce many events at once so mobile apps can produce events after an offline period. [x] As an **engineer**, I want good client libraries to produce events so that I don't have to write them myself. [x] As an **engineer**, I want client libraries that can query for sampling settings and other schema topic usage metadata. [x] As an **engineer/analyst**, I want to be able to trigger events in a developer mode, regardless of sampling settings, so I can more easily debug problems. [] As an **engineer**, I want to restrict usages of my event topic via authentication and authorization so that I don't get garbage data from internet trolls. [] As an **analyst**, I want alarms for implausible value combinations (if field A has value X, then field B has to be > 200, or such) (this might not be part of event intake, more likely it is stream processing)

As part of Modern Event Platform, we plan to build a scalable event stream intake system that will allow us to accept validated events from both internal and external clients. The working name of this component will be Stream Intake Service (subject to change). This system needs to be reliable and horizontally scalable. It should accept events synchronously (with validation responses, etc.) and asynchronously. It should accept events via HTTP POST. This component also includes any client libraries we might build that interact with the Event Intake Service as part of this component. See also {T225237} # User Stories [draft] -[x] As an **engineer**, I want to produce valid events from internal networks so production features can use event systems. -[x] As an **engineer**, I want to produce valid events from external clients so we can do comprehensive analysis based on events. -[x] As an **engineer**, I want to produce events and get an appropriate response if my event is produced so I can build production backend features. -[x] As an **engineer**, I want to produce events without a response if my event is produced so I collect "fire and forget" analytics data. -[x] As an **engineer**, I want to batch produce many events at once so mobile apps can produce events after an offline period. -[x] As an **engineer**, I want good client libraries to produce events so that I don't have to write them myself. -[x] As an **engineer**, I want client libraries that can query for sampling settings and other schema topic usage metadata. -[x] As an **engineer/analyst**, I want to be able to trigger events in a developer mode, regardless of sampling settings, so I can more easily debug problems. -[] As an **engineer**, I want to restrict usages of my event topic via authentication and authorization so that I don't get garbage data from internet trolls. -[] As an **analyst**, I want alarms for implausible value combinations (if field A has value X, then field B has to be > 200, or such) (this might not be part of event intake, more likely it is stream processing)