Change Details

Context. This is for tracking Claudia's part of the work, with Diego, on evaluating the quality of the recommendations with moderators in the moderator dashboard. Diego has built an interface. Diego is seeking how to ask the right questions to know which is better. Description. Claudia to fill in what shape this work is going to take, once that's known. Estimated Effort. Claudia to fill in, once this is known. Priority Claudia to fill in. We should make this a subtask of Diego's Phab task for Q4 on this project. I didn't know if he is creating a new task for this yet or continuing the Q3 task into Q4, so didn't do this.

**Context** Under WE 1.3 New moderator homepage T402632, Diego has built T417442 WE1.3.5 Article similarity model (ASM). The Moderator Tools team now wants to see if patrollers find the recommendations from the article similarity model better (more engaging, more interesting, more urgent) than their existing methods for finding edits for review, in order to guide how the team implements the model and presents recommended edits for review, to end-users of the Personal Dashboard. **Description** This will be a moderated participant interview with a stimulus, in the form of a list of links to relevant edits, generated by the ASM. Our anticipated participant profile is: * Active patroller of their wiki (an editor who seeks out edits by others to review them for quality) * Speaks English We plan to aim for about 4 participants from enwiki and possibly 4 participants from wikis other than enwiki; the latter group is dependent on our ability to generate a list of recommendations for non-enwiki projects. Each session should take the form of a short interview, asking participants how they currently find edits for review and their experience with these workflows, then providing the participant with a list of ASM-generated edits and asking participants to compare their existing method against the suggestions. The primary research goal of this study is to find out whether or not patrollers consider ASM-generated suggestions better than their existing workflow for finding edits to review. We also want to know what we should do in order to make ASM-generated recommendations "better" for patrollers (as compared to their existing workflows). Final deliverable should be product recommendations for the Moderator Tools Team, along with feature suggestions for the ASM. **Estimated Effort.** - For now, @Samwalton9-WMF + team will lead recruitment efforts. - @diego will need to provide a set of edits per-participant session - @cwylo will be responsible for designing the study, conducting it, and analyzing results. **Priority** High. The team anticipates the outcome of this work is needed to achieve their Q4 objective; these recommendations make up a large portion of the dashboard, and if users are not interested in it or don't view it as useful, they are unlikely to return to the dashboard. **Timeline** (See due date.)

**Context.** This is for tracking Claudia's part of the workUnder WE 1.3 New moderator homepage T402632, Diego has built T417442 WE1.3.5 Article similarity model (ASM). The Moderator Tools team now wants to see if patrollers find the recommendations from the article similarity model better (more engaging, more interesting, more urgent) than their existing methods for finding edits for review, with Diegoin order to guide how the team implements the model and presents recommended edits for review, on evaluating the qualityto end-users of the recommendations with moderators in the moderator dashboard.Personal Dashboard. **Description** Diego has built an interface.This will be a moderated participant interview with a stimulus, in the form of a list of links to relevant edits, generated by the ASM. Our anticipated participant profile is: Diego is seeking how to ask the right questions to know which is better. Description.* Active patroller of their wiki (an editor who seeks out edits by others to review them for quality) Claudia to fill in what shape this work is going to take,* Speaks English We plan to aim for about 4 participants from enwiki and possibly 4 participants from wikis other than enwiki; the latter group is dependent on our ability to generate a list of recommendations for non-enwiki projects. Each session should take the form of a short interview, asking participants how they currently find edits for review and their experience with these workflows, then providing the participant with a list of ASM-generated edits and asking participants to compare their existing method against the suggestions. The primary research goal of this study is to find out whether or not patrollers consider ASM-generated suggestions better than their existing workflow for finding edits to review. once that's knownWe also want to know what we should do in order to make ASM-generated recommendations "better" for patrollers (as compared to their existing workflows). Estimated EffortFinal deliverable should be product recommendations for the Moderator Tools Team, along with feature suggestions for the ASM. **Estimated Effort.** - For now, @Samwalton9-WMF + team will lead recruitment efforts. Claudia to fill in- @diego will need to provide a set of edits per-participant session - @cwylo will be responsible for designing the study, conducting it, once this is knownand analyzing results. **Priority** Claudia to fill inHigh. The team anticipates the outcome of this work is needed to achieve their Q4 objective; these recommendations make up a large portion of the dashboard, and if users are not interested in it or don't view it as useful, they are unlikely to return to the dashboard. We should make this a subtask of Diego's Phab task for Q4 on this project. I didn't know if he is creating a new task for this yet or continuing the Q3 task into Q4, so didn't do this.**Timeline** (See due date.)