We should be setting both server side and client side timeouts on all requests made to the elasticsearch cluster. While digging through hive logs related to a recent cluster instability problem (T142506) I found that requests made through CirrusSearch\Searcher::findNamespace don't have any explicit timeout set, it eventually times out after 5 (!!!) minutes. This should be fixed, and we should double check the rest of the places that make requests to ensure everything has an appropriate timeout. We could set a default client side timeout in CirrusSearch\Connection::__construct but we also want to set the server side timeout (with the 'timeout' query parameter).

Might also be worth looking into if we can set default server side timeouts more reasonably as well.