Let's switchover es1023 to es1024 which is already running Buster and Mariadb 10.4.
The idea is to move writes to es4 for a few minutes and then do the switchover on es5 master.
Steps:
[x] Give weight 50 to es1024
[x] Disable alerts on es5 hosts
[x] switchover.py --timeout=15 --only-slave-move es1023.eqiad.wmnet es1024.eqiad.wmnet
[x] Disable puppet es1024 and es1023
[x] Merge puppet change to promote es1024 to master https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/607236/
[x] Disable writes for es5 on MW: https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/606663/
[x] !log Starting es5 failover from es1023 es1024
[x] ./switchover.py --skip-slave-move es1023 es1024
[x] Depool es1023 entirely
[x] First 10.4 master, so let's double check that all the slaves are well connected.
[x] Let es1023 replication stopped, as it will be running 10.1 and its master will run 10.4, let's reimage before starting replication.
[x] dbctl --scope eqiad section es5 set-master es1024
[x] Enable and run puppet at es1023 and es1024
[x] events_coredb_master.sql on the new master es1024
[x] events_coredb_slave.sql on the new slave es1023
[x] Revert the above patch to make es5 writable again.
[x] Change es5-master DNS https://gerrit.wikimedia.org/r/c/operations/dns/+/609899/
[x] Disable notifications on es1023
[x] Reimage es1023
[] Slowly repool es1023 and remove weight from es1024
@jcrespo I would appreciate a review of the above simplified steps
**Date & time: Tuesday 7th July at 05:00 AM UTC**