The second round of testing was conducted using the same test protocol as written in T265531 with a single Concept B design. The reason for this test is mainly to test a proposed solution to an issue encountered in previous Add links mobile and desktop usability testing, whereby users tended to review link suggestions based on the content inside of the suggestion card (link anchor text, article title, article extract) over considering the article context.
Goal(s)
- Primary goal: Evaluate how effective highlighting the sentence around a suggested link helps with focusing users on judging links in article context.
- The hypothesis being that this helps users to read the evaluate the links in the article context over focusing too narrowly on card contents as they have tended to do in previous testing rounds.
- Since this was the main goal, and we wanted to see how this might work in a similar mobile platform, we decided to iterate on Desktop concept B (since this used the similar mobile VE mode presented inside the SE module on the homepage).
- Example highlighted sentence around suggested "cast iron" link
- Secondary goal: Evaluate minor usability changes
- Consider other smaller changes made based on findings from round 1 desktop tests.
Testing format
Due to time and personnel constraints, this will firstly be an unmoderated remote task-based test, conducted using UserTesting.com to recruit participants and record sessions in English only.
Testers
Respondents who are new to editing on Wikipedia.
Test protocol
Follows the same protocol as concept B in https://docs.google.com/document/d/1CoXAZWaa4odOSy1csEtuM5ZX0SSJuG9AMcUiACy0yro, but with a new prototype link https://1fbt6x.axshare.com/#id=z0depr&c=1
Results summary
Key finding:
Four out of five participants read at least some of the highlighted sentence during their evaluation, more so in comparison to previous mobile and desktop tests.
Two participants paid particular close attention to the other similar words in the sentence and paragraph around the suggested link, as intended in the design.
Recommendations
- Add sentence highlighting to the design (Mobile & Desktop) - whilst a couple of people did tend to start focusing on the card content, most at least started off with reviewing some suggestions with the article context in mind.
- Keep the simpler Suggested edits hover card state (Desktop only)
- Remove the extra “Suggested links” text in the Suggested edits module task explanation. (Mobile & Desktop)
- Update the onboarding info to include the clarification that anchor text ≠ article title
- Update the review summary link text style to no longer be Accent50, but the “read-only” Base20 colour.
- Revise Yes/No button designs incorporating icons and flexibility for long translated strings (Mobile & Desktop)
Full document:
https://docs.google.com/document/d/1R2Ea6v9l60oPPf3XqkGLp2pGFoD57cpBIwnjxmfFDAc/