This task involves the work of conducting an initial round of desktop usability tests of the first Edit Check we are developing: a check that prompts people to explicitly decide whether the new content they are attempting to add warrants a reference or not.

Note: usability testing of the mobile UX happened in T327356.

Decisions to be made

• 1. What – if any – aspects of the proposed desktop reference check user experience (T329579) will we revise prior to offering Edit Check in production (T338907)

Learning Objective(s)

What ambiguities/points of confusion/holes/etc. do people encounter when attempting to navigate the reference check experience and publish the edit(s) they arrived to make?

• Where "attempting to navigate" in this context refers to doing things like:
  • Knowing what they are being asked to decide
  • Knowing what content they are being asked make a decision about
    • Choosing not to add a citation to the content they are being asked about
    • Choosing to add a citation to the content they are being asked about
      • Generating a citation
      • Inserting said citation
  • Proceeding to publish the edit they will have just "finished" making
  • Moving "forward" and "backward" within the edit flow
    • E.g. "Say you notice a typo while deciding whether the content you're adding warrants a reference or not. How would you intuitively think to go back to edit this text?"

Test protocol

Edit Check (desktop) – Unmoderated Research Protocol

Recruitment filters

Test #1

• People who:
  • Read Wikipedia with some regularity (e.g. monthly/weekly)
  • Speak English as a second language
  • Living in/from a non-Western context (e.g. outside of Norther America, Canada, UK, Australia)
  • Have some experienced editing Wikipedia

Test #2

• People who:
  • Read Wikipedia with some regularity (e.g. monthly/weekly)
  • Speak English as a second language
  • Living in/from a non-Western context (e.g. outside of Norther America, Canada, UK, Australia)
  • Do NOT have experience editing Wikipedia

Findings

Positive

• All participants were able to complete & navigate all the steps of the experience
• All participants found the Edit Check flow natural in their editing experience
• Users have an understanding that content on Wikipedia needs to verifiable and reliable

Concerns

• A few participants did not appropriately follow the steps from the protocol, directly following their “intuition” or “habit” of editing and publishing their work
• Due to the limitations of the study & prototype, some users chose NOT to add a citation

Surprising

• All participants added information to the Edit Summary
• Some participants chose to edit with the Wikitext editor before returning to the visual editor
• Some participants decided to add the citation on their own, rather than doing it from the Edit Check experience – in the case of those who went faster than the steps provided by the protocol

Done

• Usability tests are run
• Findings are documented on this ticket
• Findings are published on mediawiki.org: https://www.mediawiki.org/wiki/Edit_check
  • @ppelberg: I'm marking this task as resolved without having yet done the above assuming this ticket will naturally resurface and serve as the reminder I need to do so.