Page MenuHomePhabricator
Create Task
Maniphest T267011

Analyze results of A/B test for new search location
Open, MediumPublic
Actions

Description

Description

In T261648: Perform A/B test for search location change, we performed an A/B test of the new location of the search bar in hopes that we will create a better search experience for our users. This task related to the success of this experiment

Analysis Criteria

  • Which group has a higher rate of search sessions initiated? How does this differ per wiki? How does this differ per levels of activity of the editors (nice to have).
  • Which group has a higher rate of search sessions completed? How does this differ per wiki?
  • Have any other interesting trends emerged?
  • (for logged-out users) Is there any perceived changes in search behavior before/after the change?

Related Objects
Search...

Event Timeline

ovasileva created this task.Nov 2 2020, 12:22 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 2 2020, 12:22 PM
ovasileva triaged this task as Medium priority.Nov 2 2020, 12:22 PM
ovasileva added a subtask: T265333: End A/B test for search location.
• Hermann added a parent task: T267016: Plan for november meeting with Göteborgs museer.Nov 2 2020, 1:21 PM
DannyS712 removed a parent task: T267016: Plan for november meeting with Göteborgs museer.Nov 2 2020, 1:25 PM
MNeisler added a project: Product-Analytics.Nov 2 2020, 1:43 PM
ovasileva closed subtask T265333: End A/B test for search location as Resolved.Nov 5 2020, 4:03 PM
kzimmerman moved this task from Triage to Current Quarter on the Product-Analytics board.Nov 5 2020, 6:59 PM
MNeisler edited projects, added Product-Analytics (Kanban); removed Product-Analytics.Nov 20 2020, 4:05 PM
ovasileva moved this task from Backlog to Analyst Consultation on the Readers-Web-Backlog (Tracking) board.Dec 3 2020, 6:16 PM
MNeisler moved this task from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.Dec 8 2020, 6:02 PM
MNeisler moved this task from Doing to Needs Review on the Product-Analytics (Kanban) board.Tue, Dec 22, 1:30 PM
MNeisler added a subscriber: mpopov.

@mpopov Here is the first draft of the Search AB Test analysis report for review.

The report currently includes my initial analysis for search sessions initiated and search sessions completed using some of the methodologies we discussed.
Let me know if you have any questions and thanks in advance for taking the time to review!

I'm also still working on the following changes and will provided an updated report with those changes and any needed revisions following break.

  • Summary of key findings
  • Trends for logged-out users
  • Other interesting trends
mpopov added a comment.Wed, Dec 23, 11:46 PM

Megan and I met on Dec 22 to go over initial feedback and discuss a few changes to the analysis.

MNeisler moved this task from Needs Review to Doing on the Product-Analytics (Kanban) board.Mon, Jan 4, 4:40 PM

Quick update - I'm currently revising the report based on feedback from @mpopov. I'll post an updated report for final review later this week.

ovasileva added a comment.Mon, Jan 4, 5:20 PM

Thanks @MNeisler!

Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL