Page MenuHomePhabricator

Add throttling and backoff to suggestion fetching script
Closed, ResolvedPublic

Description

Since we won't be making batched requests (T232230) due to the limitations of the Cloud Vision API, we should have throttling and backoff mechanisms in place to ensure that we don't exceed the request quotas.

https://cloud.google.com/vision/quotas

Event Timeline

Mholloway triaged this task as Medium priority.

Change 545664 had a related patch set uploaded (by Mholloway; owner: Michael Holloway):
[mediawiki/extensions/MachineVision@master] Add throttling support for the label fetcher script

https://gerrit.wikimedia.org/r/545664

Change 545664 merged by jenkins-bot:
[mediawiki/extensions/MachineVision@master] Add throttling support for the label fetcher script

https://gerrit.wikimedia.org/r/545664

Still thinking through a backoff mechanism in the case of 429 errors.

Change 545891 had a related patch set uploaded (by Mholloway; owner: Michael Holloway):
[mediawiki/extensions/MachineVision@master] Add backoff and retry support when fetching suggestions from script

https://gerrit.wikimedia.org/r/545891

Change 545891 merged by jenkins-bot:
[mediawiki/extensions/MachineVision@master] Add backoff and retry support when fetching suggestions from script

https://gerrit.wikimedia.org/r/545891

Throttling was removed yesterday to avoid getting blocked on review of the related dependency, but can be added again later if required.