This task represents the work involved with evaluating the usability of Version 2.0 of the Reply tool with people who have little-to-no-experience using Wikipedia talk pages.
This task is a a follow up to the usability test we ran with Version 1.0 of the Reply tool.
- v1.0 usability test script: Usability test of Reply v1.0
- v1.0 usability test findings: Version_1.0_prototype
Test goals
- To understand whether the new features introduced in version 2.0 of the Replying tool are intuitive for people who have little-to-no-experience using Wikipedia talk pages.
- To understand whether the Reply tool as a whole is intuitive for people who have little-to-no-experience using Wikipedia talk pages.
- To identify parts of the Replying tool that could be improved to make it easier for people to start using and to complete what they set out to accomplish.
Test details
- Platforms: desktop
- Account type: logged in and logged out
- Text input default: source and visual
- Variant A: test participants will be shown the tool's source mode when they click the "Reply" link for the first time
- Variant B: test participants will be shown the tool's visual mode when they click the "Reply" link for the first time
- Test pages: each test participant should be linked to a separate talk page.
- Each talk page link should include ?dtvisual=1 at the end of the URL to ensure the visual mode is available to test participants.
Workflow to test
In addition the Reply tool's core workflows (e.g. draft and publish a response in that discussion, locate published response on the talk page), we would like to test these new workflows Version 2.0 is introducing:
- Style the content of the comment they are writing (e.g. bold and italicize).
- Ping another user. Scenarios:
- Ping someone who has commented in the discussion the tester is commenting in
- Ping someone who has commented in a different discussion on the talk page the tester is commenting on
- Ping someone who has not commented on the talk page the tester is commenting on
- Change the user the tester had selected to ping, to a different user
- Read: if they initially had selected USER A to ping have them ping USER B
- Delete the ping they had entered into the text input
- Insert a link to another Wikipedia page into the comment they are writing
- Be made aware when someone responds to something they've said
Test script
To be added by @iamjessklein
- Each test participant should be explicitly asked what they understand the visual and source affordances to mean and do.//
Done
- Draft test script
- Run test
- Document test findings and recommendations on ticket, complete with standout quotes and links to the parts of the tests where the quotes occur.