Page MenuHomePhabricator

Gerrit: Alert when # of active threads > some threshold
Closed, InvalidPublic

Description

In gerrit we allocate 32 threads for requests. In practice, the number of active threads doesn't seem to spike above 10, unless there is a stuck lock which (currently) means we need to restart.

Judging from past instances, a threshold of 35 active threads triggering an alert that the problem is, indeed, T224448 would likely not cause false alarms and would be helpful for folks who have permissions to restart gerrit, but not experience restarting gerrit.

Event Timeline

Aren't we already alerting on Gerrit responding to HTTP requests in a timely fashion? Shouldn't catching that symptom that be enough?

thcipriani closed this task as Invalid.Tue, Sep 3, 4:07 PM

Aren't we already alerting on Gerrit responding to HTTP requests in a timely fashion? Shouldn't catching that symptom that be enough?

We are alerting on HTTP/SSH problems. The document you linked presents a compelling argument. Closing as invalid.