Page MenuHomePhabricator

Create Loading Overlay user testing plan
Closed, ResolvedPublic

Description

Write a testing script to evaluate how effective the overlay is in managing contributor expectations when there is slow performance.

Designer: @iamjessklein
Design Review: @alexhollender
Engineering Review: @matmarex
QA Review: @Ryasmeen
PM Review: @DannyH

Event Timeline

These tests can probably be a bit more of a casual ask for (qualitative) feedback as we know that this will objectively improve the experience. It's an expected behavior and a heuristic that we identified for success.

On a high level, we want to know: is this better than before?

cc/ @Esanders @Nirzar @Ryasmeen @dchen - what do you think?

Agreed - we had this raised as an issue in our previous user tests, so if it becomes a non issue in subsequent tests that should be counted as a success.

Let's do this as a one question test on usertesting.com

Step 1. ask user to look at the loading overlay
Step 2. ask user to "describe what you are looking at and what you expect to happen next."

In order to do this we will need a prototype or something for them to look at.
@matmarex - should this be a working prototype or would you prefer that I use a one page mockup?

Documenting a real time conversation with @matmarex - we should test an animated gif of the mockup (because it would be very challenging for user to get the conditions to work perfectly to recreate slow loading). I will post that here once I complete T211595

iamjessklein renamed this task from Design usability test for overlay to User Testing Plan for overlay .Feb 12 2019, 2:38 PM

Testing Plan

We will solicit, process and prioritize feedback through the following channels

AudienceFeedback channelChannel ownerTranslatorFeedback outputPrioritization
Editing TeamGoogle Chat@iamjesskleinProgram + Eng. ManagerPhabricator@iamjessklein @marcella @ppelberg
WMF DesignSlack@iamjesskleinProgram + Eng. ManagerPhabricator@iamjessklein @marcella @ppelberg
wikisMediaWiki project page*@ppelbergProgram + Eng. ManagerPhabricator@iamjessklein @marcella @ppelberg

Definitions

  • Feedback: bugs, qualitative feedback
  • Audience: who we are seeking feedback from
  • Feedback channel: the communication channel through which we will be seeking feedback
  • Channel owner: the person on the Editing Team who will facilitate the given "Feedback channel" and relate the feedback raised to Program Manager and Engineering
  • Translator: the person on the Editing Team who will translate the accumulated feedback (i.e. bugs, qualitative feedback) into Phabricator tickets
  • Feedback output: where and how the feedback we receive will be incorporated into the Editing Team's workflow
  • Prioritization: who on the Editing Team will assign priority to Loading Overlay feedback

Notes

*MediaWiki Project Page: this will be created by @ppelberg; once complete, it will be linked in the table above.

ppelberg renamed this task from User Testing Plan for overlay to Create Loading Overlay user testing plan.Feb 27 2019, 8:50 PM
ppelberg closed this task as Resolved.