Page MenuHomePhabricator

tune elasticsearch recovery settings
Closed, ResolvedPublic

Description

During our last cluster restart, the cluster failed to recover. See incident documentation for details.

To prevent this from happening again, we should probably reduce recovery_after_time to 5 minutes.

Event Timeline

Gehel created this task.Sep 21 2017, 12:59 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Gehel updated the task description. (Show Details)Sep 21 2017, 1:00 PM
debt assigned this task to Gehel.Sep 21 2017, 5:04 PM
debt triaged this task as High priority.
debt edited projects, added Discovery-Search (Current work); removed Discovery-Search.

Change 380524 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] elasticsearch: only wait 5 minutes for all nodes in case of cold restart

https://gerrit.wikimedia.org/r/380524

Change 380524 merged by Gehel:
[operations/puppet@production] elasticsearch: only wait 5 minutes for all nodes in case of cold restart

https://gerrit.wikimedia.org/r/380524

debt closed this task as Resolved.Oct 2 2017, 2:11 PM