Page MenuHomePhabricator
Create Task
Maniphest T236311

Add throttling and backoff to suggestion fetching script
Closed, ResolvedPublic
Actions

Description

Since we won't be making batched requests (T232230) due to the limitations of the Cloud Vision API, we should have throttling and backoff mechanisms in place to ensure that we don't exceed the request quotas.

https://cloud.google.com/vision/quotas

Details

ProjectBranchLines +/-Subject
mediawiki/extensions/MachineVisionmaster+71 -14Add backoff and retry support when fetching suggestions from script
mediawiki/extensions/MachineVisionmaster+60 -1Add throttling support for the label fetcher script
Customize query in gerrit

Related Objects

Event Timeline

• Mholloway created this task.Oct 23 2019, 7:16 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 23 2019, 7:16 PM
• Mholloway claimed this task.Oct 23 2019, 7:16 PM
• Mholloway triaged this task as Medium priority.
• Mholloway updated the task description. (Show Details)Oct 23 2019, 7:19 PM
gerritbot added a comment.Oct 23 2019, 9:18 PM

Change 545664 had a related patch set uploaded (by Mholloway; owner: Michael Holloway):
[mediawiki/extensions/MachineVision@master] Add throttling support for the label fetcher script

https://gerrit.wikimedia.org/r/545664

gerritbot added a project: Patch-For-Review.Oct 23 2019, 9:18 PM
gerritbot added a comment.Oct 23 2019, 10:40 PM

Change 545664 merged by jenkins-bot:
[mediawiki/extensions/MachineVision@master] Add throttling support for the label fetcher script

https://gerrit.wikimedia.org/r/545664

Maintenance_bot removed a project: Patch-For-Review.Oct 23 2019, 11:10 PM
ReleaseTaggerBot added a project: MW-1.35-notes (1.35.0-wmf.4; 2019-10-29).Oct 24 2019, 5:00 AM
MSantos moved this task from Backlog to Done on the MachineVision board.Oct 24 2019, 3:24 PM
• Mholloway moved this task from Done to In development on the MachineVision board.Oct 24 2019, 3:34 PM

Still thinking through a backoff mechanism in the case of 429 errors.

gerritbot added a comment.Oct 24 2019, 4:39 PM

Change 545891 had a related patch set uploaded (by Mholloway; owner: Michael Holloway):
[mediawiki/extensions/MachineVision@master] Add backoff and retry support when fetching suggestions from script

https://gerrit.wikimedia.org/r/545891

gerritbot added a project: Patch-For-Review.Oct 24 2019, 4:39 PM
• Mholloway moved this task from Doing to Code Review on the Product-Infrastructure-Team-Backlog (Kanban) board.Oct 24 2019, 4:39 PM
gerritbot added a comment.Oct 25 2019, 3:08 AM

Change 545891 merged by jenkins-bot:
[mediawiki/extensions/MachineVision@master] Add backoff and retry support when fetching suggestions from script

https://gerrit.wikimedia.org/r/545891

Maintenance_bot removed a project: Patch-For-Review.Oct 25 2019, 3:10 AM
• Mholloway moved this task from Code Review to To Deploy on the Product-Infrastructure-Team-Backlog (Kanban) board.Oct 28 2019, 5:23 PM
• Mholloway moved this task from In development to Done on the MachineVision board.
• Mholloway moved this task from To Deploy to Sign off on the Product-Infrastructure-Team-Backlog (Kanban) board.Nov 5 2019, 3:29 PM
• Mholloway closed this task as Resolved.Nov 8 2019, 2:07 PM

Throttling was removed yesterday to avoid getting blocked on review of the related dependency, but can be added again later if required.

Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL