I drafted a test script for review.

The test is broken down by the 5 tasks that we asked users to perform continuously:

• Add a new link
• Edit the link label
• Edit the link target
• Edit the link target with an external link
• Remove link

@iamjessklein I added questions to the "Written follow-up questions" section.

Tomorrow, I want to try the tasks with fresh eyes to make sure:
a) the test flows naturally and
b) we are asking the right prompting questions as part of each task.

Great, I've asked for a few folks on the editing team to QA the test today before we put it up on usertesting.com to double check that the flows work.

The test is up and running on usertesting.com right now.
It should take a few days to get the results in and then I will provide a synthesis after watching them.

I had to re-launch the tests because the tests were defaulting to wikitext (not Visual Editor). @Esanders has rectified that issue and have just deployed the tests. They should be done by the weekend.

The tests are complete. I will follow up by providing a synthesis here.

What flows worked:

• adding a link
• editing a link
• removing a link

What flows can be improved:

• editing a link label
• adding an external link

Suggested tweaks

• create a link label form input
• provide contextual instructions for users when searching for internal and external links
• autofill links with proper http:// or https://

Miscellaneous

• Many users still struggled to find specific content within the article. Between the lack of search and table of contents combined with a slow internet connection, it became an onerous ask for the user.
• erasing could be made easier with ⓧ buttons in the search box
• distinguishing between read and edit modes was challenging for many users.
• a few users easily missed the "remove link" button - perhaps this can be made more obvious
• error messages were completely perplexing to users
• undo button was not in an obvious location for users

cc/ @Esanders @JTannerWMF @ppelberg

Thanks for the analysis, sounds pretty successful overall.

Many users still struggled to find specific content within the article. Between the lack of search and table of contents combined with a slow internet connection, it became an onerous ask for the user.

This sounds like it could just be an artefact of our testing script. In most real world cases edits will be reactive to content they've already found, whereas in our script we say "find this specific content and change it".

@Esanders
re:

This sounds like it could just be an artefact of our testing script. In most real world cases edits will be reactive to content they've already found, whereas in our script we say "find this specific content and change it".

I agree, although I still think that it's worth noting that there are more things that we could do to make findability easier for folks when they are switching into editing mode and have long sections to navigate through. Which is, out of scope for this particular release.

Updated project page to include the findings of our usertesting.com test of Edit Cards v2: https://www.mediawiki.org/w/index.php?diff=3331428&oldid=3322809&title=VisualEditor_on_mobile%2FEdit_cards&type=revision