User story
As a product manager, I want to run this feature as an A/B test with a survey and instrumentation so that I can communicate the results and impact of this feature. =
Requirements
- Wrap up the new activity tab in an A/B test
- Randomly sort all users into groups
- 50% into A: History tab (no change)
- 50% into B: Activity tab V2
- On January 15
- Stop sorting new users into groups
- Users with group B should keep the activity tab v2 (because we plan to scale)
- Audience: all users
- Instrumentation plan created to answer the KRs within the Epic
- Feature is instrumented according to deck (may be broken up by release if needed)
Reference
Engineering notes
We should have 3 dev settings toggles to carefully control things. One is our existing feature flag toggle, then two new toggles for forcing a particular group assignment.
- Activity tab feature flag dev toggle > this enables the app to run any activity-tab-related code.
- 2 new dev toggles for forcing an assignment here, control & activity tab v2.
- When the app launches: A. If feature flag is enabled, we go through group assignment logic i. If control is forced here via dev settings toggle, assign control bucket ii. if experiment is forced here in dev settings toggle, assign experiment bucket iii. If neither are forced, go through the randomized 50/50 assignment bucket B. When setting up the root view, we check the feature flag. If not enabled, embed History. If enabled, check the group assignment, if control embed History, otherwise embed Activity tab
This means, once this task is done, to get to Activity tab we have to:
- Fresh install, go to dev settings, enable feature flag toggle, then enable forcing experiment assignment toggle.
- Terminate + relaunch the app, activity tab should show.