Page MenuHomePhabricator

Create layperson-friendly version of Media Search A/B test analysis
Closed, ResolvedPublic

Description

A few teams/people outside of the Structured Data team have taken an interest in the results of this A/B test and would like to understand the methodology and results better. The current notebook is pretty technical though. We'd like to have this in a format we can present to non-experts.

Acceptance criteria:

  • High-level explanation of the "team draft" algorithm
  • Step-by-step explanation of the query used for the analysis
  • Simplified explanation of how the data indicates preference for Media Search (and how strong that preference is)

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 19 2020, 6:18 PM
Ramsey-WMF renamed this task from Writer layperson-friendly version of Media Search A/B test analysis to Create layperson-friendly version of Media Search A/B test analysis.Oct 19 2020, 6:19 PM
LGoto triaged this task as Medium priority.Oct 27 2020, 5:08 PM

I've updated the notebook on GitHub, adding text aiming to make it more accessible to everyone.

@mpopov : It would be great if you could give it a read through to make sure my explanations of concepts are reasonable.

@Ramsey-WMF : Please give it a read and let me know if there are things that are still unclear.

It's beautiful. I have no notes. Thank you! 😺

nettrom_WMF closed this task as Resolved.Nov 16 2020, 5:04 PM

I also received word from Mikhail that he'd reviewed it and everything's good to go! Closing as resolved.