With the scalability issues we've been seeing on php-fpm when a lot of higher-latency http calls are involved, the necessity of having a proxy that can handle connections between services has become apparent.
More in general, we want to have a middleware that allows us to generically have the following capabilities, when dealing with RPC calls to other services:
- Allow connection pooling
- Work well with our DNS discovery mechanism
- Enable TLS e2e without the need for relying on every single service doing encryption the "right" way
- Allow configuring per-endpoint timeouts.
- Global and local-only rate limiting
- Allow monitoring RPC calls (telemetry and tracing)
- Tracing of RPC calls
We've evaluated nginx in the past, and the non-commercial version lacks in even the most important of these features, as it can either support dns discovery or connection pooling, not both. We already use envoy as a TLS terminator on most servers, so we can probably use it to implement such a middleware, which is also what envoy was designed for.