Add support for sending batched image annotation requests
Closed, DeclinedPublic
Actions

Assigned To

Authored By

	• Mholloway
	Sep 6 2019, 9:53 PM

Description

Currently, we're set up only for sending image annotation requests for a single image at a time.

Google Cloud Vision also provides for asynchronous batched requests of up to 2000 images at a time, as described at https://cloud.google.com/vision/docs/batch. We should update GoogleCloudVisionHandler to be able to use it. I anticipate that all of our annotation requests will ultimately use the batch API.

Note: This will require some additional setup, namely to set up a Google Cloud Storage bucket to receive results, and to configure our Handler to retreive results from it.

Open questions

What happens in case of error retrieving or labeling an image during a batched request?

Related Objects

Mentioned In: T237122: Google Cloud Vision API refuses to fetch (most?) remote URLs that do not point to Google Cloud Storage
T236311: Add throttling and backoff to suggestion fetching script

Event Timeline

• Mholloway created this task.Sep 6 2019, 9:53 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 6 2019, 9:53 PM

• Mholloway updated the task description. (Show Details)Sep 6 2019, 9:54 PM

• Mholloway updated the task description. (Show Details)

• Mholloway triaged this task as Medium priority.Sep 6 2019, 10:10 PM

• Mholloway updated the task description. (Show Details)

Moved to analysis to answer the open questions before going to implementation.

@Mholloway Kaldari says to ping him tomorrow to get the Google Cloud Storage set up.

• Jhernandez added a subscriber: MSantos.Oct 8 2019, 11:43 AM

• Mholloway claimed this task.Oct 8 2019, 3:14 PM

• Mholloway edited projects, added Product-Infrastructure-Team-Backlog-Deprecated (Kanban); removed Product-Infrastructure-Team-Backlog-Deprecated.

• Mholloway moved this task from To Do to Doing on the Product-Infrastructure-Team-Backlog-Deprecated (Kanban) board.Oct 8 2019, 3:25 PM

• Mholloway moved this task from Ready for dev to In development on the MachineVision board.Oct 15 2019, 10:11 PM

Ha: it turns out that making batched async requests actually requires putting the images to be annotated into Google Cloud Storage:

Google\ApiCore\ApiException from line 139 of /var/www/mediawiki/extensions/MachineVision/vendor/google/gax/src/ApiException.php: {
    "message": "Invalid URI provided in AnnotateImageRequest. Note that At this time, only plain GCS uris are supported and a gcs image uri has to be put in image.source.image_uri of each request.",
    "code": 3,
    "status": "INVALID_ARGUMENT",
    "details": []
}

We're not doing that.

• Mholloway mentioned this in T236311: Add throttling and backoff to suggestion fetching script.Oct 23 2019, 7:19 PM

• Mholloway mentioned this in T237122: Google Cloud Vision API refuses to fetch (most?) remote URLs that do not point to Google Cloud Storage.Nov 1 2019, 5:49 PM

Add support for sending batched image annotation requestsClosed, DeclinedPublicActions

Description

Related Objects

Event Timeline

Add support for sending batched image annotation requests
Closed, DeclinedPublic
Actions