We are exploring cost-based rate limits on media files, similar to the current shadow mode cost-based policies in the rest-gateway. In order to fine-tune policies, we would need the following data analysis.
The limits are based on response_size in kbytes and keyed to x-client-ip on a fixed-window of 1 minute.
The current limit is 600000 kbytes/minute, tallied from 0 at the beginning of the window and reset at the end.
The request that would go over-quota is let through, and the following requests during the window would be blocked.
Based on webrequest data for cache status miss and pass on upload.wikimedia.org, for half the limit 300000 kbytes/min, the limit 600000 kbytes/min and twice the limit 1200000 kbytes/min :
Client profile
- How many distinct clients would be blocked + % of total clients?
- X-Is-Browser split
- User-Agent cardinality (are there User-Agent that jump out as getting potentially blocked a lot more than others)
- res_proxy
Request profile
- Percentage of total requests blocked
- X-Is-Browser split
- res_proxy
Thanks a lot for the help!