Page MenuHomePhabricator

[SPIKE] Add Moderator module to growth experiments 4HR
Closed, ResolvedPublic

Description

In this spike for our wishathon we have demonstrated the addition of a basic module on the Homepage in GrowthExperiments extension.
We want to investigate if this is a feasible path forward for the centralized dashboard.

Acceptance Criteria:
Answers to the following:

  • Is it possible to add a moderation module to growth experiments? (yes according to our spike, but a closer look would be great to find any potential issues or limitations not addressed in the spike patch)
  • yes; we didn't get stuck on anything we tried to address in the patch
  • How to bucket people into the moderation modules?
  • the GrowthExperiments extension can bucket people deterministically using their UID as a seed. This means you can keep them in the same variant over time, even across wikis (if you use the Central User ID option). You can configure a probability for the likelyhood for users to be bucketed into a treatment or variant bucket. Arbitrary numbers of buckets & variants are supported. In the attached patch, I bucketed people in solely based on an editcount threshold defined in community config. We can also keep the module in the GrowthExperiments "control" variant and then layer in xlabs client side bucketing / experimentation on top of that. All of the questions about how to handle bucketing over time are really the same no matter where we implement the experiment (eg. if the user status changes, do you want to move them out of the treatment variant and into the control variant? Do you take features away from them then? Do you take features away at the end of a timed experiment?)
  • How to customize the existing home page to add more moderation focused items?
  • There is a primary and secondary section, which can each have multiple modules. There's not really an obvious technical limitation to how much we can add. The main concern there is UX: we don't want it to become an overwhelming pile of stuff. The current approach does a wholesale replacement of the existing modules in a "moderation mode" controlled by a toggle.
  • can we provide a structured-tasks-like help panel for diffs when a user clicks a diff link in our module?
  • The existing help panel is coupled with structured tasks. We can provide lightweight "quick tips" panels by simply injecting them into the pages with the relevant workflows. I suggest that we build this separately from the homepage. We could create a "QuickTips" extension to add these in where appropriate.

Event Timeline

DMburugu triaged this task as High priority.Jul 8 2025, 4:20 PM
DMburugu moved this task from Estimated to Kanban on the Moderator-Tools-Team board.
Kgraessle moved this task from In Progress to Ready on the Moderator-Tools-Team (Kanban) board.
jsn.sherman changed the task status from Open to In Progress.Jul 17 2025, 2:46 PM
jsn.sherman moved this task from Ready to In Progress on the Moderator-Tools-Team (Kanban) board.

Change #1170372 had a related patch set uploaded (by Jsn.sherman; author: Jsn.sherman):

[mediawiki/extensions/GrowthExperiments@master] Add moderation module to the homepage

https://gerrit.wikimedia.org/r/1170372

jsn.sherman subscribed.

Okay, I think this is worth looking at now. I just used the same "moderation monitoring" nomenclature that @Kgraessle used in her initial hackathon patch.

GE enrollment happens at account creation time. To force new all new accounts into the moderation-monitoring experiment, add this to your LocalSettings.php
$wgConditionalUserOptions['growthexperiments-homepage-variant'] = [ [ 'moderation-monitoring' ] ];

I forced an existing account by manually updating the hidden user pref in my javascript console:
new mw.Api().saveOption("growthexperiments-homepage-variant","moderation-monitoring");location.reload();

changing back to default is just as simple:
new mw.Api().saveOption("growthexperiments-homepage-variant","control");location.reload();

CC for the module may be found at Special:CommunityConfiguration/GrowthModeration
where you can configure things like enable/disable and minimum edit count

image.png (941×1 px, 68 KB)

When you are enrolled and meet the criteria, you can then see the module at Special:Homepage

image.png (744×766 px, 97 KB)

Note that I haven't documented my i18n strings or looked at tests yet.

This looks great and works as expected!

I'd be curious to see what the growth team thinks?

jsn.sherman added a subscriber: Samwalton9-WMF.

okay, per followup conversations and requests from @Samwalton9-WMF:

users that are bucketed into the moderation variant (eg. those with that meet our editcount requirements) will see a toggle that allows them to enable/disable "moderation mode" on the homepage.

disabledenabled
image.png (842×1 px, 169 KB)
image.png (845×1 px, 178 KB)

If the module is disabled in mediawiki or community config, or if the user doesn't meet the requirements, they are not put into the moderation variant. Since we have two different modes within our variant, we'll have to think carefully about measurement planning & experiment design. I think this is about as far as we can go on the tooling side until we know more about what we're trying to learn.

@Michael We'll definitely want the growth engineering folks to have a look at my updated poc patch.

jsn.sherman claimed this task.

@jsn.sherman I'm standing up this patch again as part of investigation into T403496: [SPIKE] How could we display tips to users who click through to edits or discussions from the moderator dashboard?[4H] and I'm seeing an error locally:

Failed to mount app: mount target selector "#moderation-monitoring-vue-root" returned null.

I have updated my LocalSettings.php config:

$wgConditionalUserOptions['growthexperiments-homepage-variant'] = [ [ 'moderation-monitoring' ] ];

and then ran:

new mw.Api().saveOption("growthexperiments-homepage-variant","moderation-monitoring");location.reload();

Am I missing any other setup? I definitely had this working locally at some point.

It turns out that our variant was failing to load if the help desk title wasn't set in CC; I just uploaded a new patchset that makes it optional.