T246379 resulted in a list of candidates for an API router, among them [[ https://www.envoyproxy.io | Envoy ]]. Since Envoy is already in use at the foundation, we should be particularly thorough in determining its feasibility. Where gaps exist, we should establish what sort of extensions would be required to satisfy requirements.
----
### Evaluation
For purposes of the evaluation, we created a [[ https://github.com/eevans/wmf-api-gateway | test environment ]] utilizing [[ https://docs.docker.com/compose/ | Docker Compose ]]. This environment starts an Envoy instance, and proxies requests to a [[ https://github.com/eevans/wmf-api-gateway/tree/master/echoapi | simple echo service ]]. The [[ https://github.com/envoyproxy/ratelimit | Lyft rate limiter service ]] is also started, along with an instance of Redis. A [[ https://github.com/eevans/wmf-api-gateway/tree/master/jwt | command line utility is included to generate JWTs ]], complete with custom grants, signed with RSA256.
See the projects [[ https://github.com/eevans/wmf-api-gateway/blob/master/README.md | README ]] for more.
#### Routing requests
The routes we require will generally take the form `api.wm.o/{something}/v{version}/{project}/{lang}/{path}` → `{lang}.{project}.org/{...}`; We need to parse language and project names from the source URI, and use it to construct a destination host name. This requirement is somewhat unusual; There are no standards for mapping URI elements to host names, and so unsurprisingly, Envoy has no native means of supporting this.
We were able to implement these semantics however through the use of the [[ https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/lua_filter | Lua filter ]]. In our [[ https://github.com/eevans/wmf-api-gateway/blob/master/envoy.yaml | test configuration ]], Lua is used to parse the path, string format the destination host name, and inject a custom HTTP header, `X-Internal-Host`. During routing, the [[ https://www.envoyproxy.io/docs/envoy/latest/api-v2/api/v2/route/route_components.proto.html#route-routeaction | auto_host_rewrite_header ]] option is used to substitute the destination host name, with the value of this custom header.
Despite seeming hacky, this //does work//, and even seems to be endorsed upstream for unusual requirements like these.
There may be other ways of approaching this, for example using [[ https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_conn_man/rds.html | RDS ]] to dynamically configure Envoy with generated configuration.
NOTE: It is not clear what the performance impact of using Lua to pattern match like this would be.
#### Parsing rate limit values from JWT
There is currently no requirement to perform authorization at the API router (authn will be handled upstream). However, we //are// using [[ https://en.wikipedia.org/wiki/JSON_Web_Token | JWTs ]] for all authorized requests transiting the gateway, and would like the option to leverage them to pass additional (custom) attributes in the payload. For example: Passing the rate limit from the authorization server, and using the value in calls to the rate limit service would allow us to manage/encapsulate this information with the rest of the config model.
Our [[ https://github.com/eevans/wmf-api-gateway/blob/master/envoy.yaml | test configuration ]] demonstrates that this //is// possible with Envoy. The `envoy.filters.http.jwt_authn` HTTP filter is configured to:
* Allow requests without a JWT (anonymous requests) to pass through as usual
* Validate signatures, and reject requests with tokens that have an invalid signature
* Reject requests with expired tokens
* Copy the payload of validly signed tokens into [[ https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/data_sharing_between_filters#dynamic-state | dynamic metadata ]] for access by subsequent filters
#### Rate limiting
Envoy supports both local and global rate limiting, and they can be combined so that violations of the local limit preempt a call to the global limiter. The global limiter HTTP filter communicates with a service using [[ https://en.wikipedia.org/wiki/GRPC | gRPC ]], and a [[ https://github.com/envoyproxy/ratelimit | reference implementation ]] of such a service does exist.
Ideally, we would like to parse rate limit values from the JWT payload, and pass them to the rate limiter. This is however not currently supported; The gRPC interface used between filter and service does not support the passing of a rate, and the service, as-implemented, assumes that the rate is statically configured.
As an alternative, we were able to create a [[ https://github.com/eevans/wmf-api-gateway/blob/master/envoy.yaml | configuration ]] where a class name passed via the JWT could be forwarded as a descriptor (in conjunction with the client ID) to the rate limiter service. This would require the association between classes and corresponding rates to be defined in the rate limiter configuration, but has the advantage that it does not require any code changes.
One (perhaps minor) additional issue with our test configuration is that while `envoy.filters.http.jwt_authn` allows us to put the structured JWT payload in dynamic metadata, the rate limiter filter is not able to read it. The test configuration works around this by (again) using the Lua scripting filter, this time to read values from dynamic metadata, and inject custom headers (which the rate limiter filter //can// read).
##### Possible remedies
- Code changes to the filter to enable reading the JWT payload from dynamic metadata should be a) straightforward, and b) something that upstream might be convinced to accept. With such a change, it would be unnecessary to inject custom HTTP headers using the Lua filter
- Code changes to the filter to enable passing a rate to the limiter should also be straightforward, and might also be candidates for upstreaming:
1. Modify the gRPC interface to accept an integer rate limit
1. Modify the rate limiter service to apply the limit passed in invocations
1. Modify the filter to pass a rate based on a configurable source
#### Logging
There is a [[ https://www.mediawiki.org/wiki/Core_Platform_Team/Initiatives/API_Gateway#Epic_3:_Rate_limits | requirement ]] that we maintain a count of accesses by client ID to support utilization graphs in the API portal. This should be something we can readily solve in a number of ways.
Our [[ https://github.com/eevans/wmf-api-gateway/blob/master/envoy.yaml | test configuration ]] demonstrates the use of access logging that includes a field containing the client ID:
```
[2020-04-09T21:37:03.830Z] "GET /core/v5/wikipedia/en/foo/bar/baz HTTP/1.1" 200 - 0 1389 10 0 "-" "curl/7.68.0" "9e3fc919-1c41-4f29-838c-c2c65cad174e" "en.wikipedia.org" "172.25.0.4:8888" "b6123f22-79b6-11ea-8bde-c77510c60c52"
```
NOTE: The last field (b6123f22-79b6-11ea-8bde-c77510c60c52) in the above output contains the client ID parsed from JWT payload
Additionally, Envoy access logs can also be structured as JSON, and/or delivered via a gRPC sink.
NOTE: If all else failed, the use of a filter to write directly to a backing store would always be a possibility.