Description

Hypothesis: If we rate limit requests for multimedia files based on a tally of response sizes, we will increase fairness in allocating the available bandwidth to users.

Background

This epic tracks the implementation of rate limiting based on transferred bytes for media traffic. It builds on the API rate limiting architecture and implementation work delivered in WE5.1.3 for rest-gateway, while adapting the approach to media delivery.

The key technical difference from request-count rate limiting is that the cost function is not the request itself, but response size (%BYTES_SENT%), so large transfers consume quota proportionally.

Problem statement

Current media delivery does not enforce fairness based on actual bandwidth consumption. Request-count limits are not sufficient for multimedia traffic because request sizes vary widely; a standard API request and a large video segment request consume very different resources.

We need a mechanism that applies per-client limits based on transferred bytes, without introducing more congestion or creating backpressure with bandwidth-throttling.

Scope

• Implement rate limiting based on transferred bytes for media file requests.
• Reuse and adapt existing rest-gateway rate limit configuration patterns where possible.

Out of scope

• Sliding-window rate limiting (not supported in the current reference rate limiting service implementation).

Design analysis and selected approach

Evaluated designs

1. rest-gateway reuse: Rejected
   • Route multimedia requests through rest-gateway and its existing limiter.
   • Pros:
     • Very small adaptations needed.
     • Keeps rate limit enforcement centralized.
   • Cons:
     • Routes all multimedia traffic through WikiKube.
     • Noisy neighbour potential for API routes.
2. New gateway based on the rest-gateway chart: Rejected
   • Deploy a new upload-gateway service based on the same chart as rest-gateway.
   • Pros:
     • Small adaptations needed.
     • Keeps rate limit logic centralized.
   • Cons:
     • Routes all multimedia traffic through WikiKube.
3. Configure rate limiting in swift frontends' envoy, backed by standalone ratelimit-upload: Approved
   • Enforce limits at swift frontend envoy TLS terminators, using a standalone WikiKube rate limiting service deployment (ratelimit-upload).
   • Pros:
     • Keeps the existing media traffic path unchanged while introducing rate limiting.
     • Allows for other bare-metal services to implement rate limiting.
   • Cons:
     • Duplicates rate limit logic and configuration between deployment-charts and puppet.
     • Introduces additional latency due to RTT between swift's envoy and the ratelimit-upload service.

Design decision

Designs #1 and #2 introduce an unacceptable congestion risk by routing all multimedia traffic through WikiKube, potentially impacting mission-critical services like MediaWiki.

Design #3 preserves the existing traffic path, the swift-to-rate-limiter RTT is acceptable for long media requests, and it still heavily builds upon WE5.1.3 work by porting/adapting configuration from helm templates to puppet, making the functionality available to non-WikiKube services.

Expected outcomes

• Fairer bandwidth allocation across clients for media traffic.
• Reduced impact from clients that consume disproportionate media bandwidth.
• A reusable enforcement foundation for more advanced user-aware and class-aware policies.
• Operational visibility through shadow-mode metrics and dashboarding before enforcement.

Known risks and limitations

• Post-response accounting behavior: if quota is crossed by response chunks, behavior for chunked responses needs validation (whether enforcement can interrupt mid-stream or only applies to subsequent requests). Mitigate by validating chunked response behavior during shadow mode.
• Policy tuning: initial quota and policy thresholds will require several tuning iterations to avoid over-limiting. Mitigate by using shadow mode to observe outcomes with redioscope.
• Capacity: ratelimit-upload replica count and sizing must be validated against realistic traffic. Mitigate by right-sizing during shadow mode as load on the rate limit service will be the same as when enforcing.
• Internal traffic classification: internal requests to swift need a safe exclusion mechanism. Mitigate by finding an exclusion mechanism during shadow mode.

MVP

• Rate limit key: client IP.
• Time window: fixed MINUTE.
• One initial fixed policy.
• Grafana dashboard.
• redioscope survey.

Level 1 improvements (follow-up)

• Allow edge (requestctl) to select rate limit key via x-wmf-user-id.
• Introduce policy selection based on x-is-browser score thresholds.
• Allow edge (requestctl) to select request policy via x-wmf-user-class.

Level 2 improvements (follow-up)

• Add filetype/path-based policies to support:
  • Reactive control for single-file stampedes.
  • Distinct policy classes for different media types (for example, video vs image).

Open questions

1. How does the rate limiter handle quota crossing for chunked responses when the overage occurs mid-transfer?
2. What exact initial quotas should be used for the first enforced policy?
3. What replica sizing target is required for ratelimit-upload at production load?
4. What exclusion mechanism should apply to internal swift traffic?

Dependencies

• We may need some support from @daniel for writing and using the redioscope survey.

Implementation plan and related tasks

• Deploy standalone ratelimit-upload service - T414439: Deploy a ratelimit service for upload (In Review)
• Write and test swift frontend tlsproxy configuration - T414440: Configure ms-fe envoys to use ratelimit service (WIP)
• Deploy upload rate limits in shadow-mode - T414445: Deploy upload rate limits in global shadow mode (TODO)
• Validate chunked-transfer behavior and define expected enforcement semantics - T424045: Validate chunked-transfer rate limit behavior (TODO)
• Create redioscope survey - T424051: Create redioscope survey for upload rate limit (TODO)
• Build shadow-mode analysis dashboard inspired by the API and REST Gateway dashboard - T424047: Build a dashboard for monitoring and analyzing media rate limiting behavior (TODO)

Follow-up

Tracked in T432235: [PP3.6.4] Craft and enforce rate limits for multimedia file requests based on bandwidth usage profiles

• Iterate and tune policy based on observed outcomes - T424052: Tune policy from shadow-mode data before enforcement (TODO)
• Start enforcement of rate limits - T414446: Enforce upload rate limit (TODO)