Page MenuHomePhabricator

Variant tests: explicitly store variant assignments
Closed, ResolvedPublic

Description

In our first variant test, T238888: Variant tests: "initiation" test (A vs. B), we inferred group assignments based on user's preferences and their history of changing preferences. This was possible because the thing that distinguished group A from B is whether they were initially assigned as having their suggested edits module activated -- in other words, the Variant B users started out farther down the single branch of our workflow than the Variant A users.

But for future variant tests, it won't be a single branch -- users will have different experiences and won't be able to get to the other one. We think we should explicitly store variant assignments for analysis purposes.

Event Timeline

MMiller_WMF renamed this task from Variant test: explicitly store variant assignments to Variant tests: explicitly store variant assignments.Apr 30 2020, 3:22 AM
MMiller_WMF added a project: GrowthExperiments.

Change 613161 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/GrowthExperiments@master] Provide tables for storing variant assignments

https://gerrit.wikimedia.org/r/613161

Change 613162 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/GrowthExperiments@master] Add maintenance script for adding new variant experiments

https://gerrit.wikimedia.org/r/613162

Change 613162 abandoned by Kosta Harlan:
[mediawiki/extensions/GrowthExperiments@master] Add maintenance script for adding new variant experiments

Reason:

https://gerrit.wikimedia.org/r/613162

Change 613312 had a related patch set uploaded (by Catrope; owner: Catrope):
[mediawiki/extensions/GrowthExperiments@master] Homepage: Store and manage variants

https://gerrit.wikimedia.org/r/613312

Change 613161 abandoned by Kosta Harlan:
[mediawiki/extensions/GrowthExperiments@master] Provide tables for storing variant assignments

Reason:
We will go with a user pref

https://gerrit.wikimedia.org/r/613161

Change 613312 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Homepage: Store and manage variants

https://gerrit.wikimedia.org/r/613312

Checked in betalabs - the variant assignment stored as growthexperiments-homepage-variant in user_properties table.