Objective
Run the test as described in the protocol in T273912 so that we can:
- Gauge the discoverability of notifications
- Identify improvements to the usability of topic subscriptions:
- Gauge user reactions to the notifications and expectations of how they will receive notifications
User stories
As a junior contributor, I want to subscribe to topics on wikipedia, so that I can receive notifications when people respond to the conversations that I am involved in.
As a junior contributor, I want to follow a call to action from a notification, so that I can easily engage in on-wiki conversations.
As a junior contributor, I want to unsubscribe to topics on wikipedia, so that I can stop receiving unwanted notifications.
Findings
Discoverability:
✅ 5 out of 5 successfully identified notifications in the toolbar
"I knew because it was highlighted in yellow so I clicked it and it went to the page." [UT-A]
✅ 5 out of 5 successfully located the comment they are being notified about on the talk page where it appears
"It was quite easy to locate" [UT-B]
Usability:
✅ 5 out of 5 successfully subscribed to notifications
✅ 4 out of 5 successfully unsubscribed to notification via Notification Dropdown
✅ 5 out of 5 successfully unsubscribed to notification via inline button
"It's quite intuitive" [UT- D]
"Overall this was pretty smooth." [UT-F]
Miscellaneous Observations:
✏️ 1 tester was completely unaware that they unsubscribed by double clicking the inline button
✏️ 1 tester attempted to unsubscribe via Preferences -> Notifications
✏️ 1 tester who successfully unsubscribed from the dropdown, followed the notification link to preferences and got completely confused
✏️ No testers commented on the lack of subscribe buttons on the USER TALK page.
✏️ Several testers struggled with having two separate entry points for Notifications on the toolbar
✏️ One tester was testing in a low bandwidth environment and was unsure if the test was loading or if it crashed.
✏️ One tester was testing in a low bandwidth environment and was unsure if the test was loading or if it crashed.
✏️ Several testers were confused by the copy on the Reply button
✏️ Although testers completed the unsubscribe from notification dropdown task, I am still unsure of the discoverability of this button without prompting
💡 Tester Idea: Styling threaded comments: “ it's a little interesting how there's indenting to show the threads. Maybe if there was a line or something it might break up the threads better." [UT-F]
💡 Tester Idea: Subscriptions Portal: "You know what I'd suggest: If there could be a place where I essentially know where my subscriptions are what I'm subscribed to and I can easily subscribe and unsubscribe from there." [UT-A]
Recommendations:
Key: P0=Must fix ------ P3=Minor improvement, NA = change that is tangential to the release of the main feature
[PO] In future tests consider testing for true discoverability of unsubscribe feature
[NA] Create one entry point for notifications T142981
[NA] Consider creating a Subscriptions management feature T273342
[NA] Consider styling threaded comments to improve legibility T282269
[NA] Consider styling subscribe/unsubscribe as buttons from OOUI T282261
Requirements
- testing environment created on wiki via patch demo
- usertesting.com protocol is set up for 5 tests
- 5 tests are run in English
- Findings are documented on the ticket (including notable quotes)
- If applicable, tickets are filed for issues that surface