Page MenuHomePhabricator

api-gateway helm chart: rest routes should return retry-after when a rate limit applies.
Closed, ResolvedPublic

Description

It's good practive to send a Retry-After header along with 429 (or 503) status. We can achieve this in Envoy by setting the following in the rate limit configuration:

enable_x_ratelimit_headers: DRAFT_VERSION_03
response_headers_to_add:
  header:
    key: Retry-After
    value: "%RESP(x-ratelimit-reset)%"

However, if we dobn't want the x-ratelimit-* headers to be present in the response, we will probably have to add another filter to remove them again. Or maybe there's another way to generate Retry-After?

Event Timeline

I don't see a particular issue with having the x-ratelimit-* headers in the response, but if there is one, I don't think there's another way of setting Retry-After to x-ratelimit-reset value except, like you said, to add and then remove the x-ratelimit-* headers.

daniel updated the task description. (Show Details)

I would say the Retry-After is a bit better approach. What I don't like with Envoy's x-ratelimit-*` headers is that they expose a little bit too much.

The Retry-After is simple feedback - we cannot handle your request, but you can come back after some amount of time, and we should be able to handle it.
Where Envoy returns information - what is the window size, - how much before next window reset, - what is the per window allowance.

It doesn't hurt us, but if someone wants to keep wasting our resources or be just a "bad actor" -> we're giving them all information immediately.

After thinking about it a bit, I tend to agree with @pmiazga
Adding Traffic for their opinion

Change #1224937 had a related patch set uploaded (by Daniel Kinzler; author: Daniel Kinzler):

[operations/deployment-charts@master] rest-gateway: generate retry-after header for rate-limited requests

https://gerrit.wikimedia.org/r/1224937

daniel changed the task status from Open to In Progress.Jan 9 2026, 12:30 PM
daniel claimed this task.
daniel added a subscriber: Hokwelum.

Change #1226827 had a related patch set uploaded (by Daniel Kinzler; author: Daniel Kinzler):

[operations/deployment-charts@master] rest gateway: include a meaningful body with 429 responses

https://gerrit.wikimedia.org/r/1226827

Change #1224937 merged by jenkins-bot:

[operations/deployment-charts@master] rest-gateway: generate retry-after header for rate-limited requests

https://gerrit.wikimedia.org/r/1224937

Change #1226827 merged by jenkins-bot:

[operations/deployment-charts@master] rest gateway: include a meaningful body with 429 responses

https://gerrit.wikimedia.org/r/1226827