This task track the work of "Phase 2" of the parent task T268699.

Research goals & questions

Main objective

• Evaluate task feasibility and attractiveness to new contributors.
  • 1. Are participants able to confidently confirm matches based on the suggestions and data provided?
  • 2. How accurate are participants at evaluating suggestions? And how does the actual aptitude compare to their perceived ability in evaluating suggestions?
  • 3. How do participants feel about the task of adding images to articles this way? Do they find it easy/hard, interesting/boring, rewarding/irrelevant?

Secondary objectives

• Evaluate information quality.
  • 4. What information do participants find most valuable in helping them evaluate image and article matches?
  • 5. Are participants able to write good captions for images they deem a match using the data provided?

• Identify opportunities for usability improvements
  • 6. Were there any pain points in the display of and navigation to information that hindered participants’ ability to make matches?

Target audience

The focus for this test will be those with limited experience editing on Wikipedia or Commons. Additionally, as Android will likely be the first product team to utilise this tool, the initial round of testing will be for mobile. In summary, participants characteristics sought are:

• Experience contributing: Few or no edits to Wikipedia or Commons.
• Platform: Mobile.
• Language: English – for the first round of testing. Ideally we want to replicate the study with participants in another language who are not bilingual in English to see how the limited understanding of the large proportion of English image metadata may affect results.

Testing format

The first round will be an unmoderated remote task-based test, using UserTesting.com to recruit participants and record sessions.

Test protocol:
https://docs.google.com/document/d/195EdTWYf98paQ_GiY3tF484cjhTjVBNTaQZhA0bHsFE/

Findings

Key takeaways

• General understanding of the task matching images to Wikipedia articles was reasonably good, given the minimal context provided for the tool and limited knowledge of Commons and Wikipedia editing. There are opportunities to boost understanding once the tool is redesigned in a Wikipedia UX.
• Everyone expressed interest in doing this task, and found it easy to do.
• Some participants were confused about the image quality criteria, which lead to poor judgement. This may be addressed with better guidance and context offered in the design of the task on Wikipedia.
• Overall poor caption writing that may again be drastically improved through more context and guidance in the user experience.

Full report:
https://docs.google.com/presentation/d/1g81-WljoL-aO72L24op2Fwt7Y7WsDGQesA0IQDCaZL8/