As part of the ongoing efforts to replace pybal with liberica we need to switch swift and swift-https load balancer services to IPIP encapsulation and mh-port scheduler.
Migration to IPIP can be performed per DC and the scheduler update will happen on both DCs simultaneously
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | Vgutierrez | T332027 Replace current L4LB with with Katran-based alternative | |||
| Resolved | Vgutierrez | T373020 Remove katran blockers for low-traffic non-k8s based services | |||
| Resolved | Vgutierrez | T385564 migrate swift/swift-https LB VIPs to IPIP encapsulation |
Change #1120496 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):
[operations/puppet@production] hiera,swift: Enable IPIP on ms-fe@codfw
Change #1120603 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):
[operations/puppet@production] hiera: Enable IPIP on ms-fe@eqiad
Change #1120496 merged by Vgutierrez:
[operations/puppet@production] hiera,swift: Enable IPIP on ms-fe@codfw
Mentioned in SAL (#wikimedia-operations) [2025-02-20T09:51:24Z] <vgutierrez> enabling IPIP encapsulation for swift-fe@codfw - T385564
Mentioned in SAL (#wikimedia-operations) [2025-02-20T10:16:06Z] <vgutierrez> restarting pybal on lvs2014 - T385564
Mentioned in SAL (#wikimedia-operations) [2025-02-20T10:24:11Z] <vgutierrez> restarting pybal on lvs2013, effectively enabling IPIP encapsulation for swift-fe@codfw - T385564
Change #1120603 merged by Vgutierrez:
[operations/puppet@production] hiera: Enable IPIP on ms-fe@eqiad
Mentioned in SAL (#wikimedia-operations) [2025-02-20T11:08:06Z] <vgutierrez> restarting pybal on lvs1020 - T385564
Mentioned in SAL (#wikimedia-operations) [2025-02-20T11:09:59Z] <vgutierrez> restarting pybal on lvs1019, effectively enabling IPIP encapsulation for swift-fe@eqiad - T385564
Change #1121336 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):
[operations/puppet@production] service: Switch swift and swift-https to maglev
Change #1121336 merged by Vgutierrez:
[operations/puppet@production] service: Switch swift and swift-https to maglev
Mentioned in SAL (#wikimedia-operations) [2025-02-20T11:41:02Z] <vgutierrez> restarting pybal on lvs2014 - T385564
Mentioned in SAL (#wikimedia-operations) [2025-02-20T11:43:54Z] <vgutierrez> restarting pybal on lvs2013, effectively switching swift-fe@codfw to maglev - T385564
Mentioned in SAL (#wikimedia-operations) [2025-02-20T11:47:49Z] <vgutierrez> restarting pybal on lvs1020 - T385564
Mentioned in SAL (#wikimedia-operations) [2025-02-20T11:51:12Z] <vgutierrez> restarting pybal on lvs1019, effectively switching swift-fe@eqiad to maglev - T385564
services have been migrated to IPIP encapsulation and maglev successfully:
vgutierrez@lvs1019:~$ sudo -i ipvsadm -Ln -t 10.2.2.27:443 Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.2.2.27:443 mh (mh-port) -> 10.64.0.166:443 Tunnel 40 1278 55 -> 10.64.16.92:443 Tunnel 40 1276 58 -> 10.64.32.182:443 Tunnel 40 1272 63 -> 10.64.48.149:443 Tunnel 40 1255 56 -> 10.64.130.2:443 Tunnel 40 1222 56 -> 10.64.134.13:443 Tunnel 40 1251 45 vgutierrez@lvs1019:~$ sudo -i ipvsadm -Ln -t 10.2.2.27:80 Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.2.2.27:80 mh (mh-port) -> 10.64.0.166:80 Tunnel 40 1 0 -> 10.64.16.92:80 Tunnel 40 1 0 -> 10.64.32.182:80 Tunnel 40 0 0 -> 10.64.48.149:80 Tunnel 40 2 0 -> 10.64.130.2:80 Tunnel 40 1 0 -> 10.64.134.13:80 Tunnel 40 3 0
vgutierrez@lvs2013:~$ sudo -i ipvsadm -Ln -t 10.2.1.27:443 Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.2.1.27:443 mh (mh-port) -> 10.192.0.87:443 Tunnel 40 975 115 -> 10.192.0.139:443 Tunnel 40 930 100 -> 10.192.16.76:443 Tunnel 40 958 106 -> 10.192.16.194:443 Tunnel 40 978 114 -> 10.192.32.36:443 Tunnel 40 1027 102 -> 10.192.48.44:443 Tunnel 40 971 111 vgutierrez@lvs2013:~$ sudo -i ipvsadm -Ln -t 10.2.1.27:80 Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.2.1.27:80 mh (mh-port) -> 10.192.0.87:80 Tunnel 40 1 0 -> 10.192.0.139:80 Tunnel 40 1 0 -> 10.192.16.76:80 Tunnel 40 2 0 -> 10.192.16.194:80 Tunnel 40 2 0 -> 10.192.32.36:80 Tunnel 40 1 0 -> 10.192.48.44:80 Tunnel 40 3 0