Define criteria for human editor verification of MV-generated metadata for purposes of model refinement
Closed, InvalidPublic
Actions

Assigned To

None

Authored By

	• Mholloway
	Jul 5 2019, 4:30 PM

Description

It is likely that labeling providers will provide an API for submitting human verification data in order to refine the labeling model.

How many editors must approve a label before it is considered verified? Only the uploader, or more?
What percentage of human reviewers shown the label must agree that a given label applies?

Status	Assigned	Task
Resolved	• Ramsey-WMF	T225964 [SDC] Build a depicts tag suggestion tool that is powered by machine vision platforms
Resolved	• Mholloway	T226119 Build middleware to utilize machine vision API for structured data on commons depicts tag suggestion tool
Invalid	None	T227341 Define criteria for human editor verification of MV-generated metadata for purposes of model refinement

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 5 2019, 4:30 PM

• Mholloway updated the task description. (Show Details)

• Mholloway triaged this task as Medium priority.Jul 17 2019, 3:52 PM

I think it's safe at this point to deprioritize third-party model feedback relative to other tasks.

This will depend on the provider/API. For now all we need to do is record votes.