Page MenuHomePhabricator

Audit paths for making requests to elasticsearch to ensure they have timeouts set.
Closed, ResolvedPublic


We should be setting both server side and client side timeouts on all requests made to the elasticsearch cluster. While digging through hive logs related to a recent cluster instability problem (T142506) I found that requests made through CirrusSearch\Searcher::findNamespace don't have any explicit timeout set, it eventually times out after 5 (!!!) minutes. This should be fixed, and we should double check the rest of the places that make requests to ensure everything has an appropriate timeout. We could set a default client side timeout in CirrusSearch\Connection::__construct but we also want to set the server side timeout (with the 'timeout' query parameter).

Might also be worth looking into if we can set default server side timeouts more reasonably as well.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Change 306306 had a related patch set uploaded (by EBernhardson):
[WIP] Ensure timeouts are always used

Change 306306 merged by jenkins-bot:
Ensure timeouts are always used