Page MenuHomePhabricator

api-gateway helm chart: use JWT to determine rate limit for rest routes
Closed, ResolvedPublic

Description

The rate limiting for the rest gateway should be based on trusted information. The user's identity and rate limiting class should be based on information from the JWT (the sub and rls fields, compare T399198#11006680).

Since we need the ability to fall back to an anonymous identity (using the IP address as the user ID), it is probably necessary to use a Lua filter in the Envoy config to process the data from the JWT. To achieve this, we can use Envoy's "dynamic meta-data" feature to plumb the JWT payload through to the Lua code, where it can then be used to inject artifical request headers for use by the rate limiting configuration.

For the JWT provider config we can use the one already present in the config for use with api.wikimedia.org. The new JWT session cookies (T398815) use the same key pair.

Implementation sketch:

  • use payload_in_metadata in the JWT provider config to instruct Envoy to store the JWT payload.
  • use request_handle:streamInfo() and streamInfo:dynamicMetadata() to access the JWT payload in Lua
  • use the information from the metadata to generate the x-wmf-user-id and x-wmf-user-class in Lua
NOTE: perhaps we should we change x-wmf-user-class to x-wmf-user-rlc to avoid confusion with other kinds of "user classes". "rlc" stands for "rate limiting class" and is used as the paylod key in the JWT.

Event Timeline

I have tried to implement this approach in https://gitlab.wikimedia.org/daniel/rlstools/-/tree/main/environments/envoy-helm/config, but failed to make it work. I must have olverlooked some detail. The Lua code would always spit out "No JWT data!".

Note that the OAuth 2 access tokens use a different format for their sub claim. Fixing that is T399199: Update OAuth 2.0 sessions to include new JWT session data from core.

Change #1192579 had a related patch set uploaded (by Pmiazga; author: Polishdeveloper):

[operations/deployment-charts@master] api-gateway: Rest-gateway Read `user_class` and `user_id` from JWT

https://gerrit.wikimedia.org/r/1192579

Cross linking https://phabricator.wikimedia.org/T399632 - we can use the sub claim to get user_id, but we still need to define the claim name for rate limit class.

daniel changed the task status from Open to In Progress.Nov 7 2025, 12:00 PM

It looks like we may need to use a different issuer for jwt cookis, due to a lomitation in envory:

  • for cookies, we want to igore feailed (expired) tokens, while we want to reject them when they come from an Authorization header.
  • we want to be able to distinguish between session tokens and header based auth wrt rate limits.
  • to achieve thse two things, itl ooks like we need to define two "providers" in envoy, one useing the header and one using the cookie.

The docs have a note that says that two providers can't have the same issuer, at last now when using allow_missing, which we need to do.

Change #1192579 merged by jenkins-bot:

[operations/deployment-charts@master] api-gateway: Rest-gateway Read `ratelimit_class` and `user_id` from JWT

https://gerrit.wikimedia.org/r/1192579

Change #1233154 had a related patch set uploaded (by Daniel Kinzler; author: Daniel Kinzler):

[operations/deployment-charts@master] api-gateway: Re-apply "Rest-gateway Read `ratelimit_class` and `user_id` from JWT"

https://gerrit.wikimedia.org/r/1233154

Change #1233155 had a related patch set uploaded (by Daniel Kinzler; author: Daniel Kinzler):

[operations/deployment-charts@master] rest-gateway: re.apply "add support for sessionJwt cookies"

https://gerrit.wikimedia.org/r/1233155

Change #1233154 merged by jenkins-bot:

[operations/deployment-charts@master] api-gateway: Re-apply "Rest-gateway Read `ratelimit_class` and `user_id` from JWT"

https://gerrit.wikimedia.org/r/1233154

Change #1233155 merged by jenkins-bot:

[operations/deployment-charts@master] rest-gateway: re.apply "add support for sessionJwt cookies"

https://gerrit.wikimedia.org/r/1233155

Merged and deployed

Change #1237295 had a related patch set uploaded (by Daniel Kinzler; author: Daniel Kinzler):

[operations/deployment-charts@master] rest-gateway: remove suppotr for insecure user ID cookies

https://gerrit.wikimedia.org/r/1237295

Change #1237295 merged by jenkins-bot:

[operations/deployment-charts@master] rest-gateway: remove support for insecure user ID cookies

https://gerrit.wikimedia.org/r/1237295