Page MenuHomePhabricator

Crowdsourced evaluation of Upload Wizard caption/description instructions
Closed, ResolvedPublic

Description

Use Amazon Mechanical turk to A/B test different sets of instructions for eliciting captions and descriptions of pictures during the file upload workflow for Upload Wizard.

Research page: Crowdsourced evaluation of Upload Wizard instructions

Conclusions:
Instructions inviting labelers to "Add a one-line explanation of what this file represents, including only the most relevant information" about an image produced better quality captions on average than instructions inviting labelers to add a "short phrase". No differences were observed in the length of captions or descriptions produced under any of the experimental conditions (header text or instruction text). Therefore, we intend to use the phrase "one-line explanation" in the Upload Wizard instructions for eliciting captions.

Related Objects

View Standalone Graph
This task is connected to more than 200 other tasks. Only direct parents and subtasks are shown here. Use View Standalone Graph to show more of the graph.

Event Timeline

Restricted Application added a project: Wikidata. · View Herald TranscriptApr 23 2018, 8:05 PM
Capt_Swing triaged this task as Normal priority.Apr 23 2018, 8:05 PM
Capt_Swing moved this task from Staged to In Progress on the Research board.
Capt_Swing updated the task description. (Show Details)Apr 25 2018, 9:04 PM
Ramsey-WMF moved this task from Untriaged to Tracking on the Multimedia board.Apr 25 2018, 9:31 PM
Capt_Swing updated the task description. (Show Details)May 30 2018, 10:33 PM

Study is concluded. Results are posted on the research page, and crowdsourced captions are posted on a subpage.

DarTar closed this task as Resolved.Jul 28 2018, 2:12 AM
DarTar edited projects, added Research-Archive; removed Research.
DarTar moved this task from Default to Q4-FY18 on the Research-Archive board.Jul 28 2018, 2:14 AM