Page MenuHomePhabricator
Paste P48543

LVS switch move
ActivePublic

Authored by cmooney on May 25 2023, 1:20 PM.
Tags
None
Referenced Files
F37096728: LVS switch move
Jun 7 2023, 12:45 PM
F37034417: LVS switch move
May 25 2023, 1:31 PM
F37034411: LVS switch move
May 25 2023, 1:20 PM
Subscribers
None
1. deploy1002.eqiad.wmnet - lock deploys
scap lock --all "LVS maintenance in eqiad, blocking deploys T322937"
2. cumin1001 - disable pybal on lvs
sudo cumin 'lvs1019*' 'disable-puppet "Disable puppet and pyabl T322937" && systemctl stop pybal.service'
3. Wait for all existing connections to drop off on lvs1019 & roughly equivalent to be present on 1020
https://grafana.wikimedia.org/d/000000343/load-balancers-lvs?orgId=1&from=now-1h&to=now&viewPanel=4
4. Get John to do move physically
5. Check all looks ok in terms of reachability/MAC addresses/pings etc.
ip neigh show etc, ping
6. cumin1001 - re-enable pybal
sudo cumin 'lvs1019*' 'enable-puppet "Re-enable T322937" && systemctl start pybal.service'
7. Check BGP from pybal looks ok, check connections on grafana graph start showing again for lvs
https://grafana.wikimedia.org/d/000000343/load-balancers-lvs?orgId=1&from=now-1h&to=now&viewPanel=4
8. deploy1002.eqiad.wmnet - unlock deploys
hit enter