Page MenuHomePhabricator

Curator forcemerges active weekly and yearly indexes every day
Open, In Progress, Needs TriagePublic

Description

It probably shouldn't forcemerge indexes that are being actively written to.

From the docs:

Force merge should only be called against an index after you have finished writing to it. Force merge can cause very large (>5GB) segments to be produced, and if you continue to write to such an index then the automatic merge policy will never consider these segments for future merges until they mostly consist of deleted documents. This can cause very large segments to remain in the index which can result in increased disk usage and worse search performance.

Event Timeline

Change #1146728 had a related patch set uploaded (by Cwhite; author: Cwhite):

[operations/puppet@production] logstash: nest curator configuration to support multiple jobs

https://gerrit.wikimedia.org/r/1146728

Change #1146728 merged by Cwhite:

[operations/puppet@production] logstash: nest curator configuration to support multiple jobs

https://gerrit.wikimedia.org/r/1146728

Change #1146736 had a related patch set uploaded (by Cwhite; author: Cwhite):

[operations/puppet@production] logstash: add forcemerge job

https://gerrit.wikimedia.org/r/1146736

Change #1146737 had a related patch set uploaded (by Cwhite; author: Cwhite):

[operations/puppet@production] logstash: add job schedule parameter

https://gerrit.wikimedia.org/r/1146737