Page MenuHomePhabricator

[FY26-WE.1.3] User research to inform centralising contributions for moderators
Closed, ResolvedPublic

Description

Draft scope
UX Recommendations for the design and development of a centralized location for moderators, and the workflows or processes that could be most impactful to centralize there.

  • Wikipedia only
  • Editors who are not new to editing, but are new to some moderation workflows and tasks. The product may expand to include editors who are completely new to doing moderation work as well.
  • We expect to focus on a limited number of WPs to start. While we don't have a set list, here is some criteria to consider when choosing the languages for this study:
    1. Prioritize from the list of languages that we studied in the admin research (T368791).
    2. If going outside of the above list, the language should have a large enough moderator pool (based on the draft definition of moderation activity and moderator in T371865)
    3. Include English.
  • While it is not decided what the central location exactly is or how it will exactly look like (hence this research), recommendations from this research should keep the path open for a potential joining of the central location and the newcomer dashboard.

Research questions we have might include:

  • How do moderators decide what to work on when contributing to Wikimedia projects?
    • Do they find specific areas to work in, or contribute wherever they’re needed?
    • What processes and workflows do they engage in, and how do they know when contributions are required for them?
    • How do they identify and track backlogs where they have skills to contribute?
  • What kinds of edits do patrollers want to identify and revert, that they are currently unable to effectively locate?
  • Would moderators find it helpful to have a central location which can route them to moderation needs on their wiki?
    • If so, what sorts of signals or modules would they find helpful to have access to?
  • What do newer patrollers find confusing or intimidating about getting further involved in content moderation processes?
    • e.g. tooling, policy knowledge, finding edits to moderate, complex processes?
  • What are moderators discovery or retrieval needs? Are they currently doing repetitive or cumbersome tasks related to retrieval of content? (context: this information can help us, Research, for scoping experiments related to discovery needs of editors which we would like to start experimenting with (at the service of eventually this KR or similar KRs. and to implement the AI strategy for editors.)

Priority
High. The centralized location for existing editors has a high priority for FY26, under WE.1.3. Doing the related UX research prior to the start of the FY gives the Design Research team the opportunity to raise awareness about the user needs before the work for design and development of the dashboard begins. This will assure that the PM and other decision makers have the information they need about the users and can confidently proceed.

Past work
This work will build on the learnings from Who are Moderators research (FY25-SDS.1.23) T371865 and ‘Task Prioritization’ focus area research (T381045).

Context and the text of the original request

One of the central elements of the Contributor Strategy is the notion of a centralised location for editors to discover opportunities to contribute. As part of WE1.3, the Moderator Tools team plans to test this idea with a focus on moderators: can we build a compelling central location through which editors can discover moderation workflows, or tasks requiring their attention?

To decide if this is a hypothesis we should pursue, and if so what we should build, we need to do some user research to learn more about editors’ perceptions, opinions, and challenges. This research will directly impact the work the Moderator Tools and other Contributors teams do in the coming Fiscal Year.

We want to better understand how moderators decide what to work on when contributing to Wikimedia projects. Do they find specific areas to work in, or contribute wherever they’re needed? What processes and workflows do they engage in, and how do they know when contributions are required for them? How do they identify and track backlogs where they have skills to contribute?

Stakeholder and sign-off on the Product end:
@SonjaPerry (while Sam is away)

Details

Due Date
Jun 6 2025, 12:00 AM

Event Timeline

leila renamed this task from Centralising contribution opportunities to [FY26-WE.1.3] User research to inform centralising contributions for moderators.Feb 28 2025, 1:04 AM
leila moved this task from Backlog to FY2024-25-Research-April-June on the Research board.
leila edited projects, added: Research (FY2024-25-Research-April-June); removed: Research.
leila added subscribers: • Easikingarmager, Bethany.

@Samwalton9-WMF thanks for capturing this task. I moved it to our team's Q4 lane for us to consider it for prioritization and delivery during upcoming Q4.

leila triaged this task as High priority.Feb 28 2025, 1:21 AM
leila added a project: Design-Research.
leila updated the task description. (Show Details)
leila set Due Date to Jun 6 2025, 12:00 AM.
leila added subscribers: • Isaac, cwylo.

@Samwalton9-WMF I expanded that task description. Please review and please help improve the "scope" section. The more information we have from you in that section, the better we can prepare async before talking with you more in March.

leila updated the task description. (Show Details)

Updated the task description after talking with Sam today (thanks for that, Sam). Working on our end for resourcing it. I will update it here in the coming days.

Claudia and one of our contractors will work on this task.

Next steps:
Claudia will schedule a call with Sonja for the week of March 31st to finalize scoping.

Something to think about before the meeting:
Claudia brought up that the current scopes too many possible user types. Given the timelines for this research there will be a need to reduce the scope in some way and that may include reducing the different user types that may be considered. Another option would be to focus the questions that we want an answer for.

We've met for an initial scoping meeting, to discuss some additional aspects of the research. Due to overlapping OOO, we have agreed to deliver final written reports by Jun 5 (the due date on this ticket), but we will be scheduling a share-out presentation and final public updates after Jun 11.

We've additionally gone over the scoping for this research, and narrowed down some aspects:

  • Focus on potential moderators, to help them discover potential new actions
  • Looking at wikis that are larger, more active, and have a relatively large number of monthly active admins (likely using the same >20 MAA cutoff as the Admin Recruitment Retention and Attrition study), under the assumption that larger wikis have more moderation needs and also a higher degree of moderation specialization
  • Looking at wikis where moderators have access to custom third-party tools to assist them in moderation, to understand what kinds of community-created work-queues, discoverability tools and prioritization features exist

We have not yet settled on the specific partner wikis, but hope to do so by the end of the week.

We are continuing research brief scope specification.

Update: working on the discussion guide and recruitment screener.

Update: Using the Design Research Participant Database, we've done our first round of recruitment. We just need to finish a run-through of the testing protocol on our Userlytics platform, and hope to send screeners and start scheduling sessions by end of week.

Update: We have begun our first sessions for the topic, with three more scheduled.

Update: We're at 3 sessions complete, 3 more scheduled; if all goes well we will have hit our target session count.

We've since learned that our user testing platform for this research, Userlytics, does not permit moderated sessions without the use of a webcam. We have devised a workaround with our existing suite of software for testing, but this will be useful to note for future research held on that platform.

Update: Our final two sessions are scheduled, and we have begun analysis of the sessions. The Userlytics workaround we developed (creating an unmoderated version of our testing protocol, conducting the interview portion via Google Meets) worked very well, so we may want to keep this in mind in the future for participants with no webcam.

We have successfully completed all participant sessions!

Update: we are nearing completion on the written deliverables as agreed upon at kickoff. We're hoping to schedule a share-out sometime in the next week and to update our Meta-Wiki page at that time as well.

Our Meta-Wiki page is now updated and we have completed our share-out. Moving to sign-off column.