Page MenuHomePhabricator

[L] Wrap up Navigation Round 1 changes in A/B test
Closed, DeclinedPublic

Description

NOTE: Task only necessary if we launch this as an A/B Test
Background

We are planning to measure if we saw a 50% increase in app menu donations as measured by A/B test or pre-post analysis after moving the donate option as a part of Navigation refresh part 1.

Task
  • Create a data controller in WMFData which outputs whether or not the user see the navigation changes as a whole (50% see changes, 50% do not)]
  • Wrap up Settings UI item removals and profile entry points in Explore and Article by first checking WMFNavigationDataController to confirm they are a part of the test. Do not remove Settings items if they are in control; do not display profile entry point buttons if they are in control.
Engineering notes

Consider if this will be wrapped up in an experiment or not (lean on current WMFExperimentsDataController for assigning and persisting buckets).

  1. Add new navigation experiment enums to Experiment, PercentageFileName, BucketFileName, and BucketValue enums in WMFExperimentDataController.
  2. Create a WMFNavigationDataController with two simple public methods, one for assigning the navigation experiment and one for pulling the navigation experiment assignment (control or test).
  3. Add an WMFExperimentsDataController property to WMFNavigationDataController. Within WMFNavigationDataController public methods lean on WMFExperimentsDataController for determining and persisting assignments:
    1. To assign an experiment call something like try experimentsDataController.determineBucketForExperiment(.profileNavigation, withPercentage: 50)
    2. To check experiment assignment, call something like return experimentsDataController.bucketForExperiment(.profileNavigation) == navigationProfileTest
  4. Call experiment assignment public method in WMFNavigationDataController somewhere in app launch sequence.
  5. Wrap up UI changes & profile entry points (Settings, Explore, Article) by first checking WMFNavigationDataController to confirm they are a part of the test.

Event Timeline

Something we'll want to know is if we should build in an end date to the experiment like we did with Alt Text, so that the UI switches back to the old way once the end date has passed.

Tsevener renamed this task from Create a data controller in WMFData for Navigation A/B test to [M] Create a data controller in WMFData for Navigation A/B test.Sep 12 2024, 5:22 PM
Tsevener renamed this task from [M] Create a data controller in WMFData for Navigation A/B test to [L] Wrap up Notifications Round 1 changes in A/B test.Sep 12 2024, 5:37 PM
Tsevener updated the task description. (Show Details)
Tsevener updated the task description. (Show Details)

@Tsevener did you mean "Navigation" not "Notifications" in the title for this one?

Seddon renamed this task from [L] Wrap up Notifications Round 1 changes in A/B test to [L] Wrap up Navigation Round 1 changes in A/B test.Sep 17 2024, 1:53 PM
HNordeenWMF added a subscriber: Seddon.

We decided as a team in iOS planning that we are not going to do an A/B test for the navigation changes at this time. Closing this out cc @Seddon