Page MenuHomePhabricator

Edit cards v2: conduct usertesting.com test
Closed, ResolvedPublic

Description

Task overview

This task involves the work involved writing, running a usertesting.com test and synthesizing the results of a qualitative test of the Edit Cards prototype.
More information about this prototype can be found in: T226679

Open questions

  • After a contributor lands back on the card after having made a change, is it clear to contributors their changes have been made successfully? Context: T225629#5255712

Test goals

"Done"

  • Test script is written
  • Test is run
  • Test results/feedback are synthesized

Event Timeline

ppelberg created this task.Apr 17 2019, 9:54 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 17 2019, 9:54 PM
ppelberg renamed this task from Edit cards: usertesting.com to Edit cards: conduct usertesting.com test.Apr 17 2019, 9:54 PM
marcella moved this task from To Triage to Q4 on the VisualEditor board.May 15 2019, 3:54 PM
ppelberg updated the task description. (Show Details)Jun 12 2019, 2:48 PM
ppelberg updated the task description. (Show Details)Jun 12 2019, 2:50 PM
ppelberg updated the task description. (Show Details)Jun 21 2019, 10:19 PM
ppelberg added a subscriber: iamjessklein.
ppelberg renamed this task from Edit cards: conduct usertesting.com test to Edit cards v2: conduct usertesting.com test.Jul 6 2019, 1:24 AM
ppelberg updated the task description. (Show Details)

I drafted a test script for review.

The test is broken down by the 5 tasks that we asked users to perform continuously:

  • Add a new link
  • Edit the link label
  • Edit the link target
  • Edit the link target with an external link
  • Remove link

@iamjessklein I added questions to the "Written follow-up questions" section.

Tomorrow, I want to try the tasks with fresh eyes to make sure:
a) the test flows naturally and
b) we are asking the right prompting questions as part of each task.

Great, I've asked for a few folks on the editing team to QA the test today before we put it up on usertesting.com to double check that the flows work.

iamjessklein updated the task description. (Show Details)Jul 9 2019, 2:56 PM
iamjessklein updated the task description. (Show Details)Jul 10 2019, 3:12 PM

The test is up and running on usertesting.com right now.
It should take a few days to get the results in and then I will provide a synthesis after watching them.

ppelberg reassigned this task from ppelberg to iamjessklein.Jul 10 2019, 3:14 PM
ppelberg awarded a token.

I had to re-launch the tests because the tests were defaulting to wikitext (not Visual Editor). @Esanders has rectified that issue and have just deployed the tests. They should be done by the weekend.

The tests are complete. I will follow up by providing a synthesis here.

What flows worked:

  • adding a link
  • editing a link
  • removing a link

What flows can be improved:

  • editing a link label
  • adding an external link

Suggested tweaks

  • create a link label form input
  • provide contextual instructions for users when searching for internal and external links
  • autofill links with proper http:// or https://

Miscellaneous

  • Many users still struggled to find specific content within the article. Between the lack of search and table of contents combined with a slow internet connection, it became an onerous ask for the user.
  • erasing could be made easier with ⓧ buttons in the search box
  • distinguishing between read and edit modes was challenging for many users.
  • a few users easily missed the "remove link" button - perhaps this can be made more obvious
  • error messages were completely perplexing to users
  • undo button was not in an obvious location for users

cc/ @Esanders @JTannerWMF @ppelberg

iamjessklein updated the task description. (Show Details)Jul 13 2019, 6:56 PM
iamjessklein updated the task description. (Show Details)

Thanks for the analysis, sounds pretty successful overall.

Many users still struggled to find specific content within the article. Between the lack of search and table of contents combined with a slow internet connection, it became an onerous ask for the user.

This sounds like it could just be an artefact of our testing script. In most real world cases edits will be reactive to content they've already found, whereas in our script we say "find this specific content and change it".

@Esanders
re:

This sounds like it could just be an artefact of our testing script. In most real world cases edits will be reactive to content they've already found, whereas in our script we say "find this specific content and change it".

I agree, although I still think that it's worth noting that there are more things that we could do to make findability easier for folks when they are switching into editing mode and have long sections to navigate through. Which is, out of scope for this particular release.

iamjessklein closed this task as Resolved.Jul 24 2019, 5:05 PM
iamjessklein removed a project: Editing Design.