We've enabled TCP Fastopen for a few use-cases on our public edges (at least for HTTP[S] + AuthDNS DoTLS, possibly others, DoH in the future, etc). However:
- We're not rotating the underlying TCP Fastopen server keys at all (even on an individual server basis). Basically the kernel is setting a key a boot time and keeping it forever. This can be problematic, as it allows remote parties to amass a large set of valid cookies over time.
- We're not synchronizing the keys across cluster members (within a DC, or globally in the case of GeoDNS failover and/or anycast), which means we sometimes lose opportunities for TFO to work when a client's next connection lands on a different server than before due to loadbalancing, GeoDNS, and/or anycast-routing changes.
We need a solution that both rotates the TFO keys on a reasonable schedule (and gracefully overlaps validity), and also keeps the rotated keys in sync across a given cluster.