Tool Name: pageviews
Quota increase requested: +6 pod
Reason: We appear to have exhausted our 4 pod limit. See comments at T301649 for the research that led to this conclusion.
As far back as data goes we've had no problems, except for a few days of 499s starting Feb 3, then the big and ever-growing jump since Feb 10, and you see it go down after I increased the kubernetes pod replica count to 4. I'd like to increase that more, but it appears 4 is the maximum.
This is likely due to an increase in automated traffic. (see also T226688: Block web crawlers from accessing Cloud Services). Even that shouldn't normally be a problem, though, since most bots seem to be headless or don't make XMLHttpRequests, but I could be wrong about that. At any rate, getting our pod count increased would seemingly help. If it means anything, Pageviews and its sister apps were once all 8 separate tools, which was a nightmare to maintain. If kept that way, we would have 8 * 4 = 32 pods total, so I hope asking for +6 to bring us to 10 is not too much. Let's assume these bots are good-faith, then we are only giving them cheaply fetched and cached data. Upstream services like the db replicas aren't put under much stress. It's just the pods that can't handle it.
Thank you as always!