Page MenuHomePhabricator

Add stats collection for observability of mjolnir daemons
Closed, ResolvedPublic

Description

Mjolnir daemons are doing things, but we have little to no observability into what it happening. Add some metrics to make it more obvious.

Bulk Daemon:

  • Time to process each batch from kafka
  • Size of batches returned by kafka?
  • Number of reported updates/missing/noop, same as cirrus

MSearch Daemon:

  • Current EMA used to decide if consuming is allowed
  • Current state of MetricMonitor is_below_threshold flag
  • Current FlexibleIterval value which determines how often we collect data from elasticsearch for EMA
  • Time to process each batch from kafka
  • Time to process each bulk request?