Page MenuHomePhabricator

Measure event lag in the realtime feed
Closed, ResolvedPublic5 Estimated Story Points

Description

O 6.KR1

In order to create a latency SLO for realtime events, we need to first measure the latency in the realtime stream.
The difference between event_created and event_published gives the latency for the event.

To do

  • Refine the realtime SDK, if needed, to capture the fields (event_created and event_published).
  • Using the SDK, create a script that captures the events for 7 days, pushing timestamps (event_created and event_published) in a db. The script does not need to run continuously for a week. We can run it once a day for several hours keeping track of offsets of events. When we kill the script, we can capture the OS interrupt and dump the offsets in a file. In the next run, we can pass the latest offsets.
  • Use the db data, calculate p50, p90, p99, p999 of latency
  • Prepare a doc to share

Acceptance criteria
Latency SLOs are defined and shared.

Event Timeline

JArguello-WMF renamed this task from Potential SLOs to [investigation] Potential SLOs.Jul 27 2023, 2:09 PM
JArguello-WMF updated the task description. (Show Details)
JArguello-WMF set the point value for this task to 5.
prabhat renamed this task from [investigation] Potential SLOs to [investigation] Measure event lag in the realtime feed.Jul 27 2023, 2:34 PM
prabhat updated the task description. (Show Details)
prabhat updated the task description. (Show Details)
prabhat renamed this task from [investigation] Measure event lag in the realtime feed to Measure event lag in the realtime feed.Aug 21 2023, 4:46 PM
prabhat updated the task description. (Show Details)
JArguello-WMF raised the priority of this task from Medium to High.Sep 21 2023, 1:28 PM