Page MenuHomePhabricator

Evaluate Envoy proxy for API gateway (and rate-limiter)
Closed, ResolvedPublic

Description

T246379 resulted in a list of candidates for an API router, among them Envoy. Since Envoy is already in use at the foundation, we should be particularly thorough in determining its feasibility. Where gaps exist, we should establish what sort of extensions would be required to satisfy requirements.


Evaluation

For purposes of the evaluation, we created a test environment utilizing Docker Compose. This environment starts an Envoy instance, and proxies requests to a simple echo service. The Lyft rate limiter service is also started, along with an instance of Redis. A command line utility is included to generate JWTs, complete with custom grants, signed with RSA256.

See the projects README for more.

Routing requests

The routes we require will generally take the form api.wm.o/{something}/v{version}/{project}/{lang}/{path}{lang}.{project}.org/{...}; We need to parse language and project names from the source URI, and use it to construct a destination host name. This requirement is somewhat unusual; There are no standards for mapping URI elements to host names, and so unsurprisingly, Envoy has no native means of supporting this.

We were able to implement these semantics however through the use of the Lua filter. In our test configuration, Lua is used to parse the path, string format the destination host name, and inject a custom HTTP header, X-Internal-Host. During routing, the auto_host_rewrite_header option is used to substitute the destination host name, with the value of this custom header.

Despite seeming hacky, this does work, and even seems to be endorsed upstream for unusual requirements like these.

There may be other ways of approaching this, for example using RDS to dynamically configure Envoy with generated configuration.

NOTE: It is not clear what the performance impact of using Lua to pattern match like this would be.
Parsing rate limit values from JWT

There is currently no requirement to perform authorization at the API router (authn will be handled upstream). However, we are using JWTs for all authorized requests transiting the gateway, and would like the option to leverage them to pass additional (custom) attributes in the payload. For example: Passing the rate limit from the authorization server, and using the value in calls to the rate limit service would allow us to manage/encapsulate this information with the rest of the config model.

Our test configuration demonstrates that this is possible with Envoy. The envoy.filters.http.jwt_authn HTTP filter is configured to:

  • Allow requests without a JWT (anonymous requests) to pass through as usual
  • Validate signatures, and reject requests with tokens that have an invalid signature
  • Reject requests with expired tokens
  • Copy the payload of validly signed tokens into dynamic metadata for access by subsequent filters
Rate limiting

Envoy supports both local and global rate limiting, and they can be combined so that violations of the local limit preempt a call to the global limiter. The global limiter HTTP filter communicates with a service using gRPC, and a reference implementation of such a service does exist.

Ideally, we would like to parse rate limit values from the JWT payload, and pass them to the rate limiter. This is however not currently supported; The gRPC interface used between filter and service does not support the passing of a rate, and the service, as-implemented, assumes that the rate is statically configured.

As an alternative, we were able to create a configuration where a class name passed via the JWT could be forwarded as a descriptor (in conjunction with the client ID) to the rate limiter service. This would require the association between classes and corresponding rates to be defined in the rate limiter configuration, but has the advantage that it does not require any code changes.

One (perhaps minor) additional issue with our test configuration is that while envoy.filters.http.jwt_authn allows us to put the structured JWT payload in dynamic metadata, the rate limiter filter is not able to read it. The test configuration works around this by (again) using the Lua scripting filter, this time to read values from dynamic metadata, and inject custom headers (which the rate limiter filter can read).

Possible remedies
  • Code changes to the filter to enable reading the JWT payload from dynamic metadata should be a) straightforward, and b) something that upstream might be convinced to accept. With such a change, it would be unnecessary to inject custom HTTP headers using the Lua filter
  • Code changes to the filter to enable passing a rate to the limiter should also be straightforward, and might also be candidates for upstreaming:
    1. Modify the gRPC interface to accept an integer rate limit
    2. Modify the rate limiter service to apply the limit passed in invocations
    3. Modify the filter to pass a rate based on a configurable source
Logging

There is a requirement that we maintain a count of accesses by client ID to support utilization graphs in the API portal. This should be something we can readily solve in a number of ways.

Our test configuration demonstrates the use of access logging that includes a field containing the client ID:

[2020-04-09T21:37:03.830Z] "GET /core/v5/wikipedia/en/foo/bar/baz HTTP/1.1" 200 - 0 1389 10 0 "-" "curl/7.68.0" "9e3fc919-1c41-4f29-838c-c2c65cad174e" "en.wikipedia.org" "172.25.0.4:8888" "b6123f22-79b6-11ea-8bde-c77510c60c52"
NOTE: The last field (b6123f22-79b6-11ea-8bde-c77510c60c52) in the above output contains the client ID parsed from JWT payload

Additionally, Envoy access logs can also be structured as JSON, and/or delivered via a gRPC sink.

NOTE: If all else failed, the use of a filter to write directly to a backing store would always be a possibility.

Event Timeline

Eevans created this task.Mar 26 2020, 12:51 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 26 2020, 12:51 AM
Eevans triaged this task as Medium priority.Mar 26 2020, 12:53 AM
Eevans updated the task description. (Show Details)
Eevans updated the task description. (Show Details)Mar 26 2020, 6:38 PM
Eevans updated the task description. (Show Details)Mar 30 2020, 7:41 PM
Eevans updated the task description. (Show Details)
Eevans updated the task description. (Show Details)Mar 30 2020, 7:45 PM
Eevans updated the task description. (Show Details)
Eevans updated the task description. (Show Details)Mar 30 2020, 7:57 PM
Eevans updated the task description. (Show Details)Mar 30 2020, 8:00 PM
Eevans updated the task description. (Show Details)Mar 30 2020, 8:39 PM
Eevans updated the task description. (Show Details)Mar 30 2020, 9:35 PM
Eevans updated the task description. (Show Details)Mar 30 2020, 9:45 PM
Eevans updated the task description. (Show Details)Mar 30 2020, 9:54 PM
Eevans updated the task description. (Show Details)Apr 9 2020, 1:01 AM
Eevans updated the task description. (Show Details)Apr 9 2020, 7:45 PM
Eevans updated the task description. (Show Details)Apr 9 2020, 9:08 PM
Eevans updated the task description. (Show Details)Apr 9 2020, 9:53 PM
Eevans closed this task as Resolved.Apr 10 2020, 3:42 PM
Eevans claimed this task.
Eevans updated the task description. (Show Details)
Eevans added a subscriber: Joe.