Finalise Envoy configuration with completed Rate Limiter
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	WDoranWMF
	Jun 9 2020, 4:35 PM

Description

Description
The configuration for the full Envoy API Gateway with Rate Limiter is production ready.

Done Criteria

Rate limiter is built into container
Configurable via values file
Ensure in case of failure, system fails open

Details

Subject	Repo	Branch	Lines +/-
Api-gateway: implement fallback for anon users in lua until envoy 1.16	operations/deployment-charts	master	+7 -2
Switch ratelimit service to V3 protocol	operations/deployment-charts	master	+2 -2
Configure ratelimiter to support authenticated/anon limits for api	operations/deployment-charts	master	+90 -21

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Resolved	• eprodromou	T235270 Wikimedia API Gateway
Open	None	T255034 Wikimedia API Gateway Long-term Use
Resolved	• Clarakosi	T254914 Finalise Envoy configuration with completed Rate Limiter
Resolved	• Clarakosi	T260591 Move rate limits configuration from route to virtual host
Resolved	• Clarakosi	T261350 Envoy ratelimit config cleanups after 1.16

Event Timeline

WDoranWMF created this task.Jun 9 2020, 4:35 PM

WDoranWMF triaged this task as Medium priority.Jun 9 2020, 4:45 PM

WDoranWMF edited projects, added Platform Team Workboards (Green); removed Platform Team Workboards (Epics).

WDoranWMF moved this task from Backlog to Tasks Ready for Estimation on the Platform Team Workboards (Green) board.Jun 10 2020, 12:21 AM

• eprodromou edited parent tasks, added: T246270: Administrator sets flat rate limit for API calls; removed: T254794: Envoy API Gateway Implementation.Jun 10 2020, 2:55 PM

hnowlan moved this task from Tasks Ready for Estimation to Next Sprint on the Platform Team Workboards (Green) board.Jul 8 2020, 4:13 PM

hnowlan claimed this task.Jul 8 2020, 4:16 PM

Change 619804 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[operations/deployment-charts@master] Configure ratelimiter to support authenticated/anon limits for api

https://gerrit.wikimedia.org/r/619804

gerritbot added a project: Patch-For-Review.Aug 12 2020, 5:45 PM

• Pchelolo claimed this task.Aug 17 2020, 5:04 PM

• Pchelolo moved this task from Next Sprint to Waiting for Review on the Platform Team Workboards (Green) board.

Change 619804 merged by jenkins-bot:
[operations/deployment-charts@master] Configure ratelimiter to support authenticated/anon limits for api

https://gerrit.wikimedia.org/r/619804

Change 620766 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[operations/deployment-charts@master] Switch ratelimit service to V3 protocol

https://gerrit.wikimedia.org/r/620766

Change 620766 merged by jenkins-bot:
[operations/deployment-charts@master] Switch ratelimit service to V3 protocol

https://gerrit.wikimedia.org/r/620766

Maintenance_bot removed a project: Patch-For-Review.Aug 17 2020, 7:11 PM

The above patches were deployed, however we can't mark this as done:

anon rate limits are not working yet. They depend on a feature @Clarakosi introduced into envoy, but that will only be available in 1.16 that we are patiently waiting for. Once we update, one line needs to be uncommented and it should start working.
authenticated rate limits depend on OAuthRateLimiter.

Moving to blocked.

The config is almost done, but until 1.16 anon rate limits do not work since there's no support for default descriptor values. We should try to workaround that.

Change 622650 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[operations/deployment-charts@master] Api-gateway: implement fallback for anon users in lua until envoy 1.16

https://gerrit.wikimedia.org/r/622650

gerritbot added a project: Patch-For-Review.Aug 26 2020, 6:56 PM

• Pchelolo moved this task from Doing to Waiting for Review on the Platform Team Workboards (Green) board.Aug 26 2020, 6:58 PM

Change 622650 merged by jenkins-bot:
[operations/deployment-charts@master] Api-gateway: implement fallback for anon users in lua until envoy 1.16

https://gerrit.wikimedia.org/r/622650

I guess I'm gonna put this into blocked, case in order to verify it all works correctly, we need everything else - JWT issuing with private claims, JWT verification with the correct key etc.

Maintenance_bot removed a project: Patch-For-Review.Aug 26 2020, 9:10 PM

WDoranWMF edited parent tasks, added: T255032: Wikimedia API Gateway Public Launch; removed: T246270: Administrator sets flat rate limit for API calls.Sep 2 2020, 9:00 PM

WDoranWMF moved this task from Blocked to Backlog on the Platform Team Workboards (Green) board.

• eprodromou edited parent tasks, added: T255034: Wikimedia API Gateway Long-term Use; removed: T255032: Wikimedia API Gateway Public Launch.Oct 22 2020, 8:16 PM

• Pchelolo closed this task as Resolved.May 21 2021, 6:08 PM

• Pchelolo reassigned this task from • Pchelolo to • Clarakosi.

• Pchelolo closed subtask T261350: Envoy ratelimit config cleanups after 1.16 as Resolved.

• Pchelolo closed subtask T260591: Move rate limits configuration from route to virtual host as Resolved.

• Pchelolo subscribed.