Change Details

Creating this task to track progress and discuss the way forward for handling server placement in codfw once the new racks in rows E/F come online, which can only support LVS-backed services that use IPIP. **New codfw cage** The background is that this is now somewhat urgent. The new switches for {T380240} were ordered without the VXLAN license for Juniper, and the cage will be a hybrid of Juniper/Nokia (due to needing the racks online sooner than we would be able to support a full Nokia setup). This rules our using VXLAN for layer-2 extension in the new cage. That decision was made after discussions at the SRE Summit which suggested that by the time the racks went live we would no longer have any services dependent on the layer-2 connectivity from LVS load-balancers. But I think we need to assess exactly where we are with that. **Current Status** I am aware that services deployed in Kubernetes PODs are not yet able to use IPIP. I believe that is being investigated under T352956, though I am not sure what stage we are at. There are some non-K8s services not using IPIP too, which are detailed in T373020. Given the fact we will soon have racks available for service placement in codfw that cannot support the L2 method I think we need to assess what this full list is and make sure we don't place any servers that rely on this in the new location. @Vgutierrez do you know if we have an easy way to work back from the list of services in T373020 to a set of host-name prefixes we can give to DC-Ops that can't be racked in the new cage?

Creating this task to track progress and discuss the way forward for handling server placement in codfw once the new racks in rows E/F come online, which can only support LVS-backed services that use IPIP. **New codfw cage** The background is that this is now somewhat urgent. The new switches for {T380240} were ordered without the VXLAN license for Juniper, and the cage will be a hybrid of Juniper/Nokia (due to needing the racks online sooner than we would be able to support a full Nokia setup). This rules our using VXLAN for layer-2 extension in the new cage. That decision was made after discussions at the SRE Summit which suggested that by the time the racks went live we would no longer have any services dependent on the layer-2 connectivity from LVS load-balancers. But I think we need to assess exactly where we are with that. **Current Status** I am aware that services deployed in Kubernetes PODs are not yet able to use IPIP. I believe that is being investigated under T352956, though I am not sure what stage we are at. There are some non-K8s services not using IPIP too, which are detailed in T373020. It looks like most of them are managed by the search team? Until we get everything migrated I think we need to build a list of hostname-types that we can't rack in the new cage to avoid any problems. @Vgutierrez you've some neat scripts in that task, is there an easy way to turn them into a list of hostnames for that list we can give to DC-ops?

Creating this task to track progress and discuss the way forward for handling server placement in codfw once the new racks in rows E/F come online, which can only support LVS-backed services that use IPIP. **New codfw cage** The background is that this is now somewhat urgent. The new switches for {T380240} were ordered without the VXLAN license for Juniper, and the cage will be a hybrid of Juniper/Nokia (due to needing the racks online sooner than we would be able to support a full Nokia setup). This rules our using VXLAN for layer-2 extension in the new cage. That decision was made after discussions at the SRE Summit which suggested that by the time the racks went live we would no longer have any services dependent on the layer-2 connectivity from LVS load-balancers. But I think we need to assess exactly where we are with that. **Current Status** I am aware that services deployed in Kubernetes PODs are not yet able to use IPIP. I believe that is being investigated under T352956, though I am not sure what stage we are at. There are some non-K8s services not using IPIP too, which are detailed in T373020. Given the fact we will soon have racks available for service placement in codfw that cannot support the L2 methodIt looks like most of them are managed by the search team? Until we get everything migrated I think we need to assess what this full list is and make surebuild a list of hostname-types that we docan't place any servers that rely on this in the new locationrack in the new cage to avoid any problems. @Vgutierrez do you know if we haveyou've some neat scripts in that task, is there an easy way to work back from theturn them into a list of services in T373020 to a set of host-name prefixeshostnames for that list we can give to DC-Ops that can't be racked in the new cageDC-ops?