Page MenuHomePhabricator

Monitor and alert on proximity to max documents per index
Open, Needs TriagePublic

Description

We hit max documents for the ecs-test indexes due to aqs rapidly logging errors connecting to cassandra. We only knew when max documents was hit due to the surge of indexing failures:

Could not index event to OpenSearch. status: 400, action:
  "_index"=>"ecs-1.7.0-5-test-2022.24"
  "error"=>
    "type"=>"illegal_argument_exception"
    "reason"=>"Number of documents in the index can't exceed [2147483519]"

We should know when an index is approaching max documents.