Page MenuHomePhabricator

Migrate thanos-web LB VIPs to IPIP encapsulation
Closed, ResolvedPublic

Description

As part of the ongoing efforts to replace pybal with liberica we need to switch thanos-web load balancer service to IPIP encapsulation and mh-port scheduler.

Migration to IPIP can be performed per DC and the scheduler update will happen on both DCs simultaneously

You can use the swift/swift-https commits as an example. The whole process can be orchestrated using the sre.loadbalancer.migrate-service-ipip cookbook:

$ sudo -i cookbook sre.loadbalancer.migrate-service-ipip --help
usage: cookbook [GLOBAL_ARGS] sre.loadbalancer.migrate-service-ipip [-h] --dc {eqiad,codfw} --role ROLE services [services ...]

Migrate existing LVS services to IPIP

    Performed steps:
    1. Asks the user to perform the required hiera changes
    2. Runs puppet on LVS and realservers
    3. Validates that realservers are able to handle incoming IPIP traffic
    4. Restarts pybal on affected loadbalancers

    Usage:
        cookbook sre.loadbalancer.migrate-service-ipip --dc codfw --role swift::proxy swift swift-https
    

positional arguments:
  services            Service(s) to be migrated

optional arguments:
  -h, --help          show this help message and exit
  --dc {eqiad,codfw}  Target datacenter. One of eqiad, codfw. (default: None)
  --role ROLE         Puppet role used by the realservers (default: None)

Cookbook owner team: Traffic

Event Timeline

Change #1122995 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] hiera,titan: Enable IPIP on thanos-(query|web)@codfw

https://gerrit.wikimedia.org/r/1122995

Change #1123000 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] hiera,titan: Enable IPIP on thanos-(query|web)@eqiad

https://gerrit.wikimedia.org/r/1123000

Change #1122995 merged by Vgutierrez:

[operations/puppet@production] hiera,titan: Enable IPIP on thanos-(query|web)@codfw

https://gerrit.wikimedia.org/r/1122995

Change #1123000 merged by Vgutierrez:

[operations/puppet@production] hiera,titan: Enable IPIP on thanos-(query|web)@eqiad

https://gerrit.wikimedia.org/r/1123000

Vgutierrez claimed this task.