Page MenuHomePhabricator compare old data sets vs latest from A/B test (June 2016)
Closed, ResolvedPublic10 Estimated Story Points


This ticket is to do a "quick" data comparison of older data collected for A/B tests (or otherwise retrieved) with the latest A/B test that was run for the languages by article count.

We have at least two valid data sets to look at: one data set from November 2015 that was collected before any changes were made to the portal page; and a data set that was collected after we launched the new search box in March 2016.

I'd like to see the comparison, using the typical measurements that we have now (click-through events, what links were clicked on, inaction on the page, etc) and compare what we saw in our last A/B test to the older data.

The goal of this comparison is to determine what the overall changes are to these metrics to show (via data) that the A/B tests are effective at helping visitors to do what they want/need to do on the portal page.

Event Timeline

debt triaged this task as High priority.Jun 22 2016, 1:59 PM
mpopov set the point value for this task to 10.Jul 26 2016, 5:32 PM

Looks good, thanks! Publish it!

To do:

  • Add link to Rmarkdown source on GitHub
  • Add link to download pre-made PDF
  • Make it not-self-contained HTML for faster viewing (images would be downloaded separately)

New URL (which allows additional reports in the future):

PDF on Commons:

Added to the analysis section on portal page:
and to weekly status update