Page MenuHomePhabricator

Log events that exceed the p999 latency in structured-data
Closed, ResolvedPublic3 Estimated Story Points

Description

OKR O6.KR1

As a part of our latency SLO work for realtime feed, we need to log events that are exceeding p999 latency. This will allow us to monitor and create alert in future.

To do

  • Add an env variable to structured-data : LatencyThreshold
  • In structured-data handlers, before events are published to the topics, check event_published - event_created. If this is > p999, log it along with event.identifier

QA (Acceptance criteria)
Able to see few logs in cloudwatch of the above type.

Event Timeline

JArguello-WMF set the point value for this task to 1.
JArguello-WMF changed the point value for this task from 1 to 3.Oct 19 2023, 1:08 PM

Hello @prabhat and @E.Enabulele, are there any other tickets for O6.KR1 "p50, p90, p99, p999 latency distribution defined for realtime feed"?