Currently our visiblity into failure to contact the search cluster isn't great. We can do:
fluorine:/a/mw-log# grep 'Search backend error' /a/mw-log/hhvm.log | grep 'Operation timed out' | wc -l
15
But it would be infinitely better if we could begin tracking this in graphite as we will be able to reference the user impacting effects of our upgrade process and general cluster health.
| Project | Branch | Lines +/- | Subject | |
|---|---|---|---|---|
| mediawiki/extensions/CirrusSearch | master | +44 -0 | Report instances of backend errors to graphite |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Gehel | T109089 EPIC: Cultivating the Elasticsearch garden (operational lessons from 1.7.1 upgrade) | |||
| Resolved | EBernhardson | T109122 CirrusSearch should send instances of Search backend error to graphite |
Change 232659 had a related patch set uploaded (by EBernhardson):
Report instances of backend errors to graphite