Page MenuHomePhabricator

api rate limiting: Assign ratelimit class based on IP range
Closed, ResolvedPublic

Description

Requests coming from certain IP ranges should be assigned a rate limit class, such as:

  • 1.2.3.* -> "cgnat" (some ISP using a NAT for all clients)

This will allow us to assign higher limits to ISPs and campuses using a NAT, but could also be used as a mechanism to exempt internal traffic or, temporarily, grant higher limits for Wikimedia events (Wikimania, hackathons).


Long term, we want a better solution for CGNATs, potentially based on edge unique cookies. Assessment of using IP ranges in the REST gateway vs. limiting by unique coooke at the edge:

Advantages:

  • Easy and quick to implement
  • Removes blocker for rollout
  • Flexible tool that can be used in the future to address Wikimedia-Events, etc

Disadvantages:

  • Not making use of the edge unique cookie, which would be more targeted
  • The allow-list needs to be maintained manually
  • The data needed to maintain it has to be extracted from the rate limit counters using redioscan on the command line. Could be exposed over http by redioscope, but that would require security vetting etc.

Event Timeline

Hmm, we should probably also figure out a way to route these to mw-api-int instead of mw-api-ext somehow. I have to think about this.

EDIT: Actually, cf parent, we need to track down what is making requests from 10.192 that are not healthchecks, and WMCS should go to mw-api-ext.

Change #1206956 had a related patch set uploaded (by Daniel Kinzler; author: Daniel Kinzler):

[operations/deployment-charts@master] rest-gateway: assign ratelimit class by network range

https://gerrit.wikimedia.org/r/1206956

daniel changed the task status from Open to In Progress.Nov 20 2025, 8:57 PM

It looks like exempting local traffic and relying on x-trusted-request: A is sufficient.

Change #1206956 abandoned by Daniel Kinzler:

[operations/deployment-charts@master] rest-gateway: assign ratelimit class by network range

https://gerrit.wikimedia.org/r/1206956

daniel updated the task description. (Show Details)
daniel edited subscribers, added: Raine; removed: SLong-WMF, hnowlan.

Re-opening this. Turns out we still need this mechanism to provide flexibility to mitigate issues with identifying clients behind CGNATs as well as for certain types of internal traffic.

Change #1244696 had a related patch set uploaded (by Daniel Kinzler; author: Daniel Kinzler):

[operations/deployment-charts@master] rest-gateway: assign ratelimit class by network range

https://gerrit.wikimedia.org/r/1244696

Change #1244696 merged by jenkins-bot:

[operations/deployment-charts@master] rest-gateway: assign ratelimit class by network range

https://gerrit.wikimedia.org/r/1244696

daniel claimed this task.