db1128 (m5 - master) needs its kernel upgraded, let's failover it to db1132
[x] Reimage db1132
[x] Move db1132 to m5 as slave
Databases on m5 (excluding labswiki, as hopefully it won't be there once this is ready to happen):
```
labsdbaccounts
mailman3
mailman3web
striker
test_labsdbaccounts
toolhub
```
When: Tuesday 23rd Nov at 14:00 UTC
Failover process
OLD MASTER: db1128
NEW MASTER: db1132
[x] Decrease m5-master TTL to 1M
[x] Check configuration differences between new and old master
`$ pt-config-diff h=db1128.eqiad.wmnet,F=/root/.my.cnf h=db1132.eqiad.wmnet,F=/root/.my.cnf `
[] Silence alerts on all hosts
[] Topology changes: move everything under db1132
`db-switchover --timeout=15 --only-slave-move db1128.eqiad.wmnet db1132.eqiad.wmnet`
[] Disable puppet db1128 and db1132
`puppet agent --disable "switchover to db1132 T288720"`
[] Merge gerrit: TO-DO
[] Run puppet on dbproxy1017 and dbproxy1021 and check the config
`puppet agent -tv && cat /etc/haproxy/conf.d/db-master.cfg`
[] Start the failover: `!log Failover m5 from db1128 to db1132 - T288720`
[] DB switchover
`root@cumin1001:~/wmfmariadbpy/wmfmariadbpy# db-switchover --skip-slave-move db1128 db1132 `
[] Reload haproxies
```
dbproxy1017: systemctl reload haproxy && echo "show stat" | socat /run/haproxy/haproxy.sock stdio
dbproxy1021: systemctl reload haproxy && echo "show stat" | socat /run/haproxy/haproxy.sock stdio
```
[] kill connections on the old master (db1128)
` pt-kill --print --kill --victims all --match-all F=/dev/null,S=/run/mysqld/mysql.sock`
[] Restart puppet on old and new masters (for heartbeat):db1128 and db1132
`puppet agent --enable && puppet agent -tv`
[] Check services affected: mailman3 toolhub striker
[] Clean orchestrator heartbeat to remove the old masters' one, otherwise Orchestrator will show lag
[] Place db1117:3325 on dbproxy as a default stand by
[] Uprade db1128's kernel
[] Restore TTL back to 5M