Page MenuHomePhabricator

Usability tests for collaborative contribution designs
Closed, ResolvedPublic

Description

Background

We want to conduct some usability tests for the collaborative contribution designs to determine how intuitive and inviting it is, as well as any confusion or challenges experienced by people. We may also want to test some concepts for later designs (such as the progress against goals) to see which concepts are the most exciting and motivating to people as well.


Goals
  1. Determine how intuitive and inviting the flow/concept is
  2. Learn if people experience confusion/challenges while navigating the flow
  3. Understand which future concepts that we have are exciting to people
Target audience

We will recruit both event/project organizers and participants to ensure we capture perspectives from those who lead collaborations and those who contribute to them. Organizers will be invited via a message posted in the Telegram group, which includes project context and a call to participate. Participants will be recruited through the Research database, with a screener question that will be used to identify those who have participated in a Wikimedia event/campaign.

Format

This will be an unmoderated remote based-task test, conducted using Userlytics to record sessions.

Testing materials

Prototypes/static screens on:

  • Associating an edit with an event
    • Checkbox before publishing (in edit summary)
      • Progress bar + statistics
    • Event mode
    • Pop-up after publishing
  • Contributions table

Full testing protocol can be found here:

Event Timeline

ifried renamed this task from [placeholder] Usability tests for collaborative contribution designs to Usability tests for collaborative contribution designs.Jul 2 2025, 6:05 PM

@JFernandez-WMF and I discussed the usability test findings so far on July 17, and we have decided on the following next steps:

  • Julieta: Share with team in Slack a short message on what we have learned so far
  • Ilana: Talk with CRS & ambassadors about getting feedback from some organizers
Learnings

We've received 4 tests from event participants so far, and this is what we've learned:

Overall, all participants found the overall concept of associating edits with events motivating and encouraging. Also, being part of a collective effort and seeing progress (toward goals) was consistently described as inspiring and rewarding.

Feedback on association methods:

  1. Checkbox in edit summary

There was mixed clarity with this one, 2 users found the wording unclear or easy to miss. The other 2 users found it clear and well-placed. There were suggestions to reword the copy to something more actionable (e.g. phrase 'This edit is part of an event' as a question, or clarify that the edit can be associated to an event that they are registered to).

  1. Event mode

Participants liked automatic association and the ability to toggle on/off. Banner links (chat, organizer) were seen as helpful and supportive, and people expressed that it created a sense of a “central hub” or shared space for the event. There was also some confusion because 2 users didn't immediately realize it meant their edits would be included in the event (it is not clarified in the banner UI), it needs clearer explanation that edits are being associated while it’s on.

  1. Post-publish pop-up

All users found the timing appropriate—it appears while their mind is still on the edit but it doesn't interrupt the flow. The wording is clear and it is a simple interaction.

Contributions tab:

All users correctly expected to find event edits in that tab, and they appreciated the layout and the functionality. There were as well some suggestions for improvement: add article links, add a 'last updated' timestamp, leaderboards.
The 'thank' feature was really well-received overall, and seen as a positive recognition. One user didn’t recognize it as an action and suggested a clearer UI (e.g., its own column).

Progress/stats:

Users appreciated seeing progress stats/goals: it made contributions feel impactful and made them as participants feel part of a community and “of something bigger.” They found them motivating. There were some suggested features: show personal progress (e.g., “How many edits have I made?”), leaderboards, milestones, or badges.

Opting out:

All participants supported the idea of having an opt-out option, even if they didn’t expect to use it, to preserve control and respect the workflow.

What's next?

We've asked organizers for their feedback, but we've learned that the Userlytics platform/interface was confusing for them and had limitations in terms of language, for example. We still have the tests running in case we receive more, but we're also going to rely on the feedback that is sent our way after we launch the project page. I am therefore closing this task, will re-open if needed.