Page MenuHomePhabricator

Personalized first day: experiments (Variation C)
Open, LowPublic

Description

Once Variation C is ready, we should plan for experiments we will do with that variation.

This will result in modifications to the written experiment plan to indicate what share of users should receive Variation A, Variation C, and no survey. That can be up to @nettrom_WMF and @SBisson.

Event Timeline

I've discussed this with both the Product Analytics team as well as @MMiller_WMF and we've decided that the experiment with Variation C will depend on the outcome of our A/B test of Variation A, specifically how the survey affects editor activation rate (which is measured in T212799).

  • If our analysis find that there is no significant difference in activation rate between Variation A and the control group, then we run a one-month experiment comparing Variation A against Variation C. If we find after that month that Variant A is worse than Variant C, then we run a subsequent test of Variant C against a control group.
  • If our analysis finds that Variant A resulted in a significant reduction in activation rate, then we run a one-month A/B test of Variation C against a control group.
nettrom_WMF moved this task from Triage to Doing on the Product-Analytics board.

Leaving this in progress because the next step is to get results from this experiment on mediawiki.org. This is not urgent.

kzimmerman triaged this task as Low priority.Wed, Sep 11, 9:42 PM