Elasticsearch 5.3.2 seems to have caused a visible perf regression on query percentiles:
- fulltext: +20ms
- morelike: +40ms
- compsuggest: +2ms
Young GC activity seems to have jumped as well while the amount of heap used seems to have decreased:
Description
Description
Event Timeline
Comment Actions
Change 358383 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] elasticsearch: remove UseConcMarkSweepGC
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2017-06-13T14:22:28Z] <gehel> restarting elasticsearch on relforge to validate GC configuration - T167636
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2017-06-13T15:09:12Z] <gehel> applying new GC configuration on elastic1018 - T167636
Comment Actions
elastic1018 is looking good, with significantly lower GC times than other nodes (see grafana). Next test is to roll out to the whole cluster...
Comment Actions
Change 358383 merged by Gehel:
[operations/puppet@production] elasticsearch: remove UseConcMarkSweepGC
Comment Actions
Change is merged. It will require a full cluster restart to be taken into account before we can actually close it. Cluster restart is planned to start on Monday June 19th.