Page MenuHomePhabricator

User research for the Task Prioritization Wishlist Focus Area
Closed, ResolvedPublic

Description

Scope

  • English Wikipedia and 1-2 more languages (with diversity of languages in terms of the size of the language)
  • Desktop users and use-cases are priority
  • Task prioritization as a high level area, with a focus on Watchlist
  • Heavier focus on moderators and patrollers as the users of Watchlist

Input

  • Wishes submitted by the community
  • Additional interviews by heavy and lighter users of Watchlist
  • Literature review done for Who are Moderators (T371865)
  • Additional light literature review for task prioritization in moderation activities on Wikipedia or other volunteer run online platforms.

Deliverable(s)

  • A report of recommendations (in the form of a Google document) for areas of investments by the Product team to respond to some of the top priority and high impact needs of Watchlist users. The report ideally also includes some insights about "task prioritization" needs by moderators. This latter sets of insights can inform work in T387462.
  • The recommendations may include recommendations for implementing specific task prioritization wishes by the community but can go beyond the specific list as well.

Dependencies

  • This work may have dependency on Product Analytics for determining stats that can inform proposed ranked tasks.

Internal documents

Deadline
April 16, 2025, to leave sufficient time in April-June for the Product team to take action on recommendations.

Context
As part of FY25-PES1.2 (Future of the Wishlist) we defined moderator focus areas in the Community Wishlist. We (Moderator Tools team) now want to do user research to understand which of the wishes and potential workstreams in the 'Task prioritization' focus area are most promising with a view to improving moderator satisfaction so that we can scope work for a FY25-WE1.3 (Increase user satisfaction in 3 moderation products).

In this year's community wishlist (https://meta.wikimedia.org/wiki/Community_Wishlist) we scoped 3 moderator-related focus areas. Of these, we think Task Prioritization is the most promising workstream for our team to pick up (https://meta.wikimedia.org/wiki/Community_Wishlist/Focus_areas/Task_prioritization). In particular it seems like we could focus on the Watchlist and consider this a tool in scope of WE1.3, which would help us make progress towards that KR. There are a number of wishes in scope of the focus area, and plenty of existing feature requests for the Watchlist. We would appreciate design research support to sort through these, interview users, and determine where the high impact opportunities are.

Details

Due Date
Apr 16 2025, 12:00 AM

Event Timeline

leila updated Other Assignee, added: dchen.
leila updated the task description. (Show Details)
leila updated Other Assignee, removed: dchen.

@Samwalton9-WMF Daisy will be in touch with you in the week of January 13 for scoping this task and I will triage the task by January 17th. If you want to move this work forward asynchronously, please feel free to start filling out some sections we have put together in the task description.

Daisy and I checked in about this task. Sam is out this week and next and can't meet. They will meet in the week of February 3rd. Given that this task's scoping will take more time, I will move it to Research-Backlog. Once it's scoped and prioritized, we can track it as part of Staged-->In progress lanes.

Daisy is meeting with Sam on February 5th. Please update this task. I will review later in the week and decide on prioritization (my understanding is that this area is high priority and I hope to be able to prioritize it with the additional information).

met with Sam and Olga this week and discussed needs and timeline.

goal of the project is to understand editors and patrollers' use of watchlist, and their highest priority pain points. the current approach is to identify a small, medium, and large wiki and recruit 5-10 editors/patrollers who use watchlist (with ranges of # pages watched) from each for moderated interviews and contextual exploration. hope is to receive a report of findings by end of march.

would like to be able to collaborate with a data analyst for 1) identifying whether there are any existing data that can illuminate or imply any watchlist-related needs and 2) identifying patrollers from the small, medium and large wikis that are TBD (suggestions from sam/olga are pending; only condition now is ideally to avoid wikis that receive the patroller survey). will reach out to Yu-ming as he is working on the moderator tools/patrollers survey and he can possibly point me in the right direction.

legal release forms have been processed through privacy@, discussion guide in progress.

there may be a pivot in focus from watchlist wish prioritization to dashboard/task prioritization. much of the discussion guide will still be relevant, but it will be tweaked pending further confirmation and information from sam.

update for the week:

current plan is to proceed with a survey that will cover basic watchlist use cases and sentiments and understanding task prioritization/collaboration. this will be used as a way to collect information for users interested in participating in follow-up interviews.

feedback on research brief and survey/interview questions pending. many discussions on how to target users and survey delivery method in progress.

leila triaged this task as High priority.Mar 5 2025, 3:35 AM
leila updated the task description. (Show Details)
leila set Due Date to Apr 16 2025, 12:00 AM.

weekly update:

  • research materials and messaging staged
  • translations of research materials pending and/or pending review
  • first batch of outreach ready to go; per Sam they will go out first thing next week. additional batches every other day
  • after translations are completed, that language wiki will receive the outreach all in one blast
  • review of Moderators report, and attempting to find any other literature regarding task prioritization in online communities (quite limited thus far)

weekly update:

  • outreach completed via massmessages and specific on-wiki gathering pages, reminders sent as appropriate
  • 7 completed EN interviews thus far, 9 additional scheduled for coming weeks
  • 2 scheduled from RO
  • 7 additional interviews completed, 1 of which was with an RO participant. outreach and reminders to RO participants have not been fruitful, and there was an RO no-show this week (have sent reschedule reminders)
  • review and synthesis of feedback ongoing
  • all interviews have been completed
  • review of moderators research and review of other sources continues to determine relevance for inclusion in watchlist reporting
  • synthesis of interviews continues; some initial feedback and mentions of existing wishes shared with Sam
  • transcripts reviews mostly completed, synthesis ongoing
  • provided some updates to Sam, Jack and Olga in terms of contained items that can tackled this quarter:
    • creating multiple watchlists (to organize topically and/or for prioritization purposes)
    • allowing users to note a 'watched for [reason]' for reference
    • adding an expiration reminder/opportunity for extension on temporarily watched pages

Finalizing the report this upcoming week!

Final report shared with team.
Research shareout tentatively scheduled for April 29.