Page MenuHomePhabricator

Dismantle experiments manager
Open, Needs TriagePublic

Description

As part of the SDS 2.1.3 goal in T360485: [EPIC] Newcomer homepage: Community updates module (FY23/24 WE1.3 / FY24/25 SDS2.1.3) it is desired to start using MetricsPlatform experiment variant assignment functionality instead of similar existing functionality in GrowthExperiments. The goal of this task is to define clear steps to dismantle the GrowthExperiments experiment management system.

Current status quo

ExperimentUserManager in GrowthExperiments is not "feature-flagged", that means that in practice any user interacting with GrowthExperiments features gets an experiment variant assigned. Even if there are no experiments in course, all users get the "control" variant assigned. In GrowthExperiments, being in the control group means getting all the enabled features for a given wiki.

GrowthExperiments assigns the experiment variant when a new local user is created. The variant is then stored in the user_property table, the relevant property is 'growthexperiments-homepage-variant'. The experiments run by Growth team across years have left a trace of variants across wikis.

Steps to dismantle the variant assignment
  1. Make GrowthExperiments not assign any experiment variant on account creation
  2. Lookup GrowthExperiments code checking for user to be in a variant, drop such logic
  3. Lookup GrowthExperiments code force setting a variant, drop such logic
  4. Lookup GrowthExperiments code expecting a variant to always be present (instrumentation at least), adapt such code to make the variant optional since only users part of an experiment will have one
  5. Lookup the rest of GrowthExperiments code depending on ExperimentUserManager and drop logic
  6. Remove 'growthexperiments-homepage-variant' user property from all wikis where it's present
  7. Remove Growth experiment management config variables: wgGEUserVariants, wgGEDefaultUserVariant, wgGEHomepageDefaultVariant, GEHomepageNewAccountVariantsByPlatform
  8. Lookup for CSS leftovers only applied to passed experiments

Affected files

Event Timeline

Sgs updated the task description. (Show Details)
Sgs updated the task description. (Show Details)

I added T400047: [EPIC] Tracking of Growth notifications CTRs using xLab (SDS 2.1.6, FY25-26) as a subtask, because figuring out whether xLab is meeting our requirements for an experimentation management system is the prerequisite to dismanteling GrowthExperiments' custom implementation.

And I'm moving this task to Blocked, pending the resolution of SDS 2.1.6.