Overview
We performed a round of restarts following https://gerrit.wikimedia.org/r/c/operations/puppet/+/632319, but we later uncovered an issue that prevented the readahead changes from taking effect. We need to restart again in order to get the readahead shrinking change properly applied.
AC
- Restart performed on all relevant elasticsearch nodes (relforge, cloudforge, eqiad, codfw)
- relforge
- cloudelastic
- eqiad
- codfw
- Verified new readahead values are taking effect
DEPLOY NOTES
We'll deploy the production cirrus clusters (eqiad + codfw) first, and then cloudelastic and relforge later.
This is because for cloudelastic100[5,6] the readahead-udev rule doesn't work due to different partition. (See https://phabricator.wikimedia.org/T265699 to track progress on that fix)