In the 2022-03-27 API incident, we found that changeprop kept connections to the API servers open for too long.
Normally those long-lived connections are good for performance -- they keep us from losing time on handshakes -- but in this case, when the load on the API servers became uneven, the persistent connections caused it to stay uneven, until we restarted Envoy on the api servers. If we set a shorter max duration, still reusing the connection but not for so long, that situation will recover on its own.
As far as I can tell this isn't currently supported in changeprop, but I expect the libraries it uses to be able to do so.