Opening this investigation task after a quick chat with @Vgutierrez
We might be able to do better than the current ping-offload servers by offloading the pings with eBPF instead on the LVS servers.
That would allow us to not require extra VMs. The ping-offload setup would also need to be re-architected (or decomed) with the new network design.
It looks like we get this for free with katran, ICMPv4 support on https://github.com/facebookincubator/katran/blob/d7575aeae1069a0f761c0414f5c30c05b50285fd/katran/lib/bpf/handle_icmp.h#L286-L288 && ICMPv6 support on https://github.com/facebookincubator/katran/blob/d7575aeae1069a0f761c0414f5c30c05b50285fd/katran/lib/bpf/handle_icmp.h#L239-L241
That's awesome
Is part of the rationale for the ping servers not to reduce traffic (as well as load) on the LVS? I don't really have an objection, but eBPF doesn't prevent a link saturating in a ping flood. Not sure if that's an issue or risk at all tbh but figured it should be part of the discussion.
Indeed, amazing ! Just a few lines of code to replace multiple VMs and router policies :)
Is part of the rationale for the ping servers not to reduce traffic (as well as load) on the LVS? I don't really have an objection, but eBPF doesn't prevent a link saturating in a ping flood. Not sure if that's an issue or risk at all tbh but figured it should be part of the discussion.
iirc the issue that we were facing is that the high number of IMCP echo was triggering Linux kernel rate limiters. Which was rate-limiting more important ICMP in the process.
Link saturation from ICMP is not a risk we're considering.