Background
The Growth team will be adding Task recommendations (a.k.a. "Suggested edits") for newcomers on their homepage this Q1/2019-20, with the hypothesis that newcomers are more likely to start and continue to edit (i.e., become "retained) under certain conditions. Namely, they are more likely to edit when:
- offered recommendations related to their interests;
- shown guidance at at the right time; and
- when given sufficient context during editing.
Goals of this user test
Overall, the aim of this task at this stage is focused on usability testing to better inform design of the UI flow as users add and make selection of tasks.
- Gauge the discoverability of adding the newcomer tasks module.
- Identify improvements to the usability of the UI (more details TBD in the document)
- Do users understand they can do X?
- Are users able to successfully do Y?
- Is there anything users that would like to be able to do?
- Gauge user reactions to the newcomer task module and expectations of how it relates to their user profile.
Testing format
UserTesting.com will be used to recruit users and run the test.
It will be a remote, unmoderated test where users will be asked go through the feature as a series of tasks, while "talking aloud" their expectations and experiences. It will be followed by a few written questions. Each session test should take no longer than 15 minutes to complete.
Testers
10 users will be recruited using UserTesting.com. The intention will be to screen for respondents who likely do not already have a Wikipedia account, and are less-to-not familiar with editing on Wikipedia.
- 5 will test on the Desktop - ideally a mix of VE and Wikitext 2017 editing UI (so that users see a version that has the "?" icon in the top editing toolbar)
- 5 will test on the Mobile Visual Editor UI
Test protocol
https://docs.google.com/document/d/1F9ZfMhP_ZCuwoCm1IGIadhe_-9ihPXgdvQCKncJ9-6k/
Findings
Desktop
https://docs.google.com/presentation/d/19XKZ8H8EWJ7G5bebQVIrgHu_4uCUwZbt1owApLTEdrE/
Mobile
https://docs.google.com/presentation/d/1ekahY6oPJg8DyKaYOTyUMnVNVaYJ3SteQMk9Xy-Ot_s/