Change Details

**Context ** The Growth team has maintained an Account creation KPI view inside its [[ https://grafana.wikimedia.org/d/bSDW0wJ7z/growth-team-product-kpis?orgId=1&from=now-7d&to=now&timezone=utc&var-tasktype=$__all&var-wiki=$__all&var-questiontype=$__all&var-variant=$__all | KPIs dashboard ]] for a long time which plots the number of new accounts registered grouped by the experiment assigned group the accounts were created. This setup has worked for a long time because it was designed to align with the bespoke GrowthExperiments enrollment system which did two very specific things (1) Enroll users only at account creation and store a MW user preference to remember it (2) Assign the `control` group to users 50% of users in an experiment but also to all the //unsampled// population of the experiment. These two assumptions are no longer true because (1) Growth wants to experiment with other audiences not just newcomers (anon, aged accounts) and (2) because TestKitchen does not assign the control variant to experiment //unsampled// users. The code that backs these charts is integrating with internals of TestKitchen that it should not use. TestKitchen will soon have an exposed API to achieve the same (getting a a deterministic enrollment result for a logged in user in different MW parts), see T413983. ** Proposed plan** The proposal would be to make use of the new API from T413983 once it's ready to convert the current //Account creation// KPI view into an //Experiments// KPI view, described as: >>! In T412942#11549192, @Michael wrote: > Could we have a simpler logic for this? > > # Loop through all the valid experiments that we have going on > # for each record experiment name and experiment group (control/treatment/unsampled) as 2 labels > # have one chart per experiment showing the distribution of the associated groups At the time of writing this there's also PRD document going on about TK adding some `Experiment#logExposure()` method that could also be interesting to mimic, or not. **Acceptance criteria ** - [] GrowthExperiments does not use TestKitchen internals - [] The Account creation KPI section is turned into Experiments section

**Context ** The Growth team has maintained an Account creation KPI view inside its [[ https://grafana.wikimedia.org/d/bSDW0wJ7z/growth-team-product-kpis?orgId=1&from=now-7d&to=now&timezone=utc&var-tasktype=$__all&var-wiki=$__all&var-questiontype=$__all&var-variant=$__all | KPIs dashboard ]] for a long time which plots the number of new accounts registered grouped by the experiment assigned group the accounts were created. This setup has worked for a long time because it was designed to align with the bespoke GrowthExperiments enrollment system which did two very specific things (1) Enroll users only at account creation and store a MW user preference to remember it (2) Assign the `control` group to users 50% of users in an experiment but also to all the //unsampled// population of the experiment. These two assumptions are no longer true because (1) Growth wants to experiment with other audiences not just newcomers (anon, aged accounts) and (2) because TestKitchen does not assign the control variant to experiment //unsampled// users. The code that backs these charts is integrating with internals of TestKitchen that it should not use. TestKitchen will soon have an exposed API to achieve the same (getting a a deterministic enrollment result for a logged in user in different MW parts), see T413983. **Proposed plan** The proposal would be to make use of the new API from T413983 once it's ready to convert the current //Account creation// KPI view into an //Experiments// KPI view, described as: >>! In T412942#11549192, @Michael wrote: > Could we have a simpler logic for this? > > # Loop through all the valid experiments that we have going on > # for each record experiment name and experiment group (control/treatment/unsampled) as 2 labels > # have one chart per experiment showing the distribution of the associated groups At the time of writing this there's also PRD document going on about TK adding some `Experiment#logExposure()` method that could also be interesting to mimic, or not. **Acceptance criteria ** - [] GrowthExperiments does not use TestKitchen internals - [] The Account creation KPI section is turned into Experiments section

**Context ** The Growth team has maintained an Account creation KPI view inside its [[ https://grafana.wikimedia.org/d/bSDW0wJ7z/growth-team-product-kpis?orgId=1&from=now-7d&to=now&timezone=utc&var-tasktype=$__all&var-wiki=$__all&var-questiontype=$__all&var-variant=$__all | KPIs dashboard ]] for a long time which plots the number of new accounts registered grouped by the experiment assigned group the accounts were created. This setup has worked for a long time because it was designed to align with the bespoke GrowthExperiments enrollment system which did two very specific things (1) Enroll users only at account creation and store a MW user preference to remember it (2) Assign the `control` group to users 50% of users in an experiment but also to all the //unsampled// population of the experiment. These two assumptions are no longer true because (1) Growth wants to experiment with other audiences not just newcomers (anon, aged accounts) and (2) because TestKitchen does not assign the control variant to experiment //unsampled// users. The code that backs these charts is integrating with internals of TestKitchen that it should not use. TestKitchen will soon have an exposed API to achieve the same (getting a a deterministic enrollment result for a logged in user in different MW parts), see T413983. ** Proposed plan** The proposal would be to make use of the new API from T413983 once it's ready to convert the current //Account creation// KPI view into an //Experiments// KPI view, described as: >>! In T412942#11549192, @Michael wrote: > Could we have a simpler logic for this? > > # Loop through all the valid experiments that we have going on > # for each record experiment name and experiment group (control/treatment/unsampled) as 2 labels > # have one chart per experiment showing the distribution of the associated groups At the time of writing this there's also PRD document going on about TK adding some `Experiment#logExposure()` method that could also be interesting to mimic, or not. **Acceptance criteria ** - [] GrowthExperiments does not use TestKitchen internals - [] The Account creation KPI section is turned into Experiments section