On https://console.cloud.google.com/apis/api/texttospeech.googleapis.com/quotas you can request an increase:

In T316009#8178345, @TheresNoTime wrote:

On https://console.cloud.google.com/apis/api/texttospeech.googleapis.com/quotas you can request an increase:

Do we want to proactively request an increase now, independent of T316006: Handle missing files?

Do we want to proactively request an increase now, independent of T316006: Handle missing files?

Sounds like we should! Moving it into sprint 33

Estimating a "requests per minute" limit in order to provide a new quota limit is fairly difficult — if, on deployment, x Phonos audio render requests are generated and added to the job queue:

• how quickly will the Job Queue attempt to run through them? [0]
• when the limit is hit, what is the starting retry delay? [1]
• how aggressively does the retry delay ramp?

Depending on the outcome of T317131: [8 hours] Investigate JobQueue and the job processing rate, I'm leaning towards picking a high number (10k?) and letting the retries handle propagating out the initial generations. I think @MusikAnimal suggested we reach out to Google and see if they'd consider giving us a "burst" quota to handle these initial spikes in (re)generation.

[0] https://grafana.wikimedia.org/d/LSeAShkGz/jobqueue?orgId=1&viewPanel=20 suggests ~220/min averaged over 5m
[1] https://wikitech.wikimedia.org/wiki/Kafka_Job_Queue#Architecture_overview

The quota increase request includes a text field marked "Your description will be sent to your service provider and is used to evaluate your request. It's useful to include the intent of the quota usage, future growth plans, region or zone spread, and any additional requirements or dependencies. " — I'm happy to write this, and would primarily be giving an overview of the Phonos project (linking to https://meta.wikimedia.org/wiki/Community_Wishlist_Survey_2022/Generate_Audio_for_IPA ?) and touching on the possibility of reaching out separately to discuss burst quotas. Is there anything specific I should mention and/or should someone else be writing such a request? Perhaps @NRodriguez or @KSiebert?

In T316009#8232966, @TheresNoTime wrote:

The quota increase request includes a text field marked "Your description will be sent to your service provider and is used to evaluate your request. It's useful to include the intent of the quota usage, future growth plans, region or zone spread, and any additional requirements or dependencies. " — I'm happy to write this, and would primarily be giving an overview of the Phonos project (linking to https://meta.wikimedia.org/wiki/Community_Wishlist_Survey_2022/Generate_Audio_for_IPA ?) and touching on the possibility of reaching out separately to discuss burst quotas. Is there anything specific I should mention and/or should someone else be writing such a request? Perhaps @NRodriguez or @KSiebert?

Maybe the partnership team could/should handle the request? They might have done similar things in the past.

In T316009#8234171, @dmaza wrote:

Maybe the partnership team could/should handle the request? They might have done similar things in the past.

Happy to hand it over to whomever 😸

I will contact them about it

I wrote Nicholas from Partnerships and will report back.

Last update from Google was 2022-10-02 — the request was still in review