Page MenuHomePhabricator

SameAs A/B test preliminary analysis
Closed, ResolvedPublic

Description

Background

We would like to take a look at the sameAs A/B test data to ensure no major negative changes have occurred since the A/B test was rolled out

Acceptance criteria

  • Answer the following questions:
  • Are pageviews in the test and control groups dramatically different?
  • Are there any other reasons for concern?
  • Generate a graph of the following:
  • pageviews per day in test and control groups for enwiki
  • pageviews per day in test and control groups for all wikis except enwiki
  • pageviews per day in test and control groups for all wikis

Event Timeline

ovasileva created this task.

Things are looking okay so far:

plot1.png (800×1 px, 215 KB)

plot2.png (800×1 px, 151 KB)

Would have been nice to see a visible positive change soon after deployment so that's a little disappointing but at least there doesn't seem to be anything bad going on. Will follow up in January in T211191

(Code & queries over at https://github.com/wikimedia-research/SEO-Experiment-SameAsProp)

@ovasileva feel free to re-open if you have additional questions