Background
We would like to test making the search bar more prominent within the article, our hypothesis is:
If we make the article search bar more prominent, we will increase the number of users who initiate searches by 8%, possibly leading to a 1% increase in search retention rate for logged out users.
Requirements
- Create logic for A/B test between current article view, and article view with search bar (T383828)
- Audience: users with French, Arabic, German, Japanese language as their primary language at the start of the test
- Assign 50% of unique users to receive control on all article views (blue magnifying glass icon), 50% of unique users receive variant on all article views (no magnifying glass, full search bar)
- At experiment end date (31 March), assignment should stop and everyone should see the control experience. At that point we will evaluate results and decide whether to roll the new search bar out to all users.
Reference
A/B test
Engineering Notes
- L creating the two variants, with no assigning logic tied to it. Considering we already have the magnifying glass variant in place, we need to create the new full bar variant (we probably have that in git history). I'm keeping it as a large, considering that the logic to handle both could be complicated.
- The magnifying glass option is done in T363606. The work done here can significantly reduce the effort, maybe to an M. I'll not change the preliminary estimation at this point, but that will be considered for the final estimation.
- TS: A lot of this work is done in the system nav bar PR, just hidden behind a flag (see needsSearchBar flag, set to true, search bar should appear on Article). We might be able to reduce or cut this task.
Notes for QA
- Fresh install app, in onboarding, choose one of the target languages from "Requirements" section above. After onboarding dismisses, you will be assigned to the control or test group. This assignment event (action: launch) is sent to the ios.search stream, but it will likely happen before you are able to enable the Send analytics to wmflabs developer settings toggle. We will have to validate this event via beta testers.
- Ensure you will sometimes (roughly half of the time) see the magnifying glass search icon on Article, and the other half you will see the Article search bar.
- Go to Developer Settings and enable Send analytics to wmflabs toggle.
- Test the events on slide 5: https://docs.google.com/presentation/d/1SWG3deIq4_ubJTheLLxaZems8vsL45ClwhpDEhrnYao/edit#slide=id.g328148da10c_0_15 Using the ios.search stream.
- (If possible) Change device date to be after experiment end date. Repeat step 1-2. For step 2 ensure you always see the magnifying glass search icon on Article if the device date is after the experiment end date.