Background

In T406808, we conducted a first round of usability testing using a Figma prototype. This allowed us to validate early assumptions about discoverability, comprehension, and expectations around Suggestion Mode. However, due to the limitations of a Figma prototype, we were not able to fully evaluate real user interactions, system behavior, or the end-to-end flow of Suggestions and Checks.

Suggestion Mode is now available in production, and we've completed an initial controlled experiment of the experience (T404600). Through this controlled experiment, we learned that while suggestions were available in more than 50% of the edit sessions newcomers and Junior Contributors initiated, people actually saw (read: clicked/tapped to view) a suggestion in 27% of those edit sessions.

In line with the above, this task involves the work of conducting a usability test to learn what (if any) facets of the UX could be improved to increase the likelihood that someone actually sees a suggestion. This will help us decide what (if any) facets of the UX we will prioritize adjusting ahead of the start of the second controlled experiment (T432427).

Decision(s) to be made

• 1. What (if any) facet(s) of the Suggestion Mode MVP experience does the Editing Team need to prioritize before or immediately after the conclusion of the second controlled experiment (T432427) to increase the likelihood that newcomers and Junior Contributors will see suggestions in sessions where ≥1 edit suggestion is available?

Usability test

DRAFT
Suggestion Mode holistic testing

Research questions

Include decisions to be made

Open questions

This section contains the questions/uncertainties that surface through designing the MVP. Prior to beginning the test, we'll decide which of these questions we'll prioritize using the usability test to help us answer.

• 1. To what extent – if any – do people become confused by the similarities between the blue text highlight which T404604#11233838 specified for suggestions and the blue text highlight which is used for existing text selections and hove effects?
  • @DLynch raised this point in T404604#11233971.

Usability testing findings

Acceptance criteria (or done)

• Implement a coded prototype
• Conduct usability test with newcomers and analyze results
• Decide on what (if any) adjustments will need to be made in Suggestion Mode