This will pick up the kernel fix for T261389.

As we're not changing the es5 master, just rebooting it, we can use a simplified version of the normal maintenance procedure.

Time: Wed, 2020-11-25, 0800 UTC (0900 CET)

Steps:

• Downtime all es5 hosts: sudo -H cookbook sre.hosts.downtime --minutes 30 -r "Reboot es1024 for kernel upgrade T268469" 'A:db-section-es5'
• Merge mediawiki-config CR to disable writes to es5 https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/643030
• Deploy MW change from deploy1001: cd /srv/mediawiki-staging/; git status; git fetch; git rebase; scap sync-file wmf-config/db-eqiad.php "Disable writes to es5 T268469"
• Check that es5 is read-only (only heartbeat update statements in mysqlbinlog)
• Stop mariadb on es1024
• Check kibana to ensure that MW is coping fine.
• Reboot es1024
• Start mariadb: systemctl start mariadb
• Disable read_only: mysql -e "set global read_only = off"
• Restart prom exporter: systemctl restart prometheus-mysqld-exporter
• Check that replication is working correctly: sudo -H db-replication-tree es1024
• Revert MW change
• Deploy MW revert from deploy1001: cd /srv/mediawiki-staging/; git status; git fetch; git rebase; scap sync-file wmf-config/db-eqiad.php "Re-enable writes to es5 T268469"
• Check that icinga is all green