The scenario we need to prevent is the following:
# User establish a TCP session to an anycasted VIP: packets are for example flowing through: `user -> transit_A -> cr1 -> server1`
# A routing change on the Internet happen and the user path becomes: `user -> transit_B -> cr2 -> server2`
This would cause the TCP session to break as server2 has no knowledge of that client.
The two possible ways for the infra to handle that use case are:
* A) `user -> transit_B -> cr2 -> server1` **consistently**
* B) `user -> transit_B -> cr2 -> cr1 -> server1`
As hashing is done on source/dest L3 headers, the `cr1->server1` step is consistent.
About A) I so far can't find any mention of it in Juniper's consistent hashing doc ([[ https://www.juniper.net/documentation/en_US/junos/topics/topic-map/switches-interface-load-balancing.html#id-configuring-consistent-load-balancing-for-ecmp-groups | here ]] and [[ https://www.juniper.net/documentation/en_US/junos/topics/topic-map/switches-interface-load-balancing.html#id-understanding-consistent-load-balancing-through-resilient-hashing-on-ecmp-groups | there ]]). Which mean we would need to test it to figure out if it works as we want it. Which is unlikely.
Regardless, Juniper consistent LB only works with single-hop eBGP, while we currently do multi-hop (as the servers peer with the routers loopback). And only on MPCs, so maybe not on MX204s.
As a side note, Juniper's consistent LB means that if the server pool is >2, adding/removing a server will not reshuffle all the sessions, which we don't need to care about right now as we only have 2 servers per sites.
However, it is possible to use BGP MEDs to achieve B). By having all servers tag a higher MED to a prefix when they advertise that prefix to the same router.