Page MenuHomePhabricator

Check in sameAs A/B test results
Closed, ResolvedPublicJan 13 2019

Description

Background

We would like to check in periodically with the sameAs A/B test data to ensure no major negative changes have occurred since the A/B test was rolled out

Acceptance criteria

  • Answer the following questions:
  • Are pageviews in the test and control groups dramatically different?
  • Are there any other reasons for concern?
  • Generate a graph of the following:
  • pageviews per day in test and control groups for enwiki
  • pageviews per day in test and control groups for all wikis except enwiki
  • pageviews per day in test and control groups for all wikis

Details

Due Date
Jan 13 2019, 11:00 PM

Event Timeline

ovasileva created this task.
Restricted Application changed the subtype of this task from "Task" to "Deadline". · View Herald TranscriptDec 5 2018, 11:05 AM
Niedzielski subscribed.

I think T211190 should be a child of this since it's the initial checkin. I've made the change. Please revert if I'm mistaken.

mpopov closed this task as Resolved.EditedJan 16 2019, 4:40 PM

Just updated my database of sampled pages (using December 2018 snapshot) and recounted pageviews from 2018-11-01 to 2019-01-15 (code & data over at GitHub). There has not been any change, up or down since the rollout:

plot1.png (800×1 px, 244 KB)

plot2.png (800×1 px, 253 KB)

plot3.png (800×1 px, 229 KB)

@ovasileva: we can probably just turn it off at this point as it's pretty clear that it's not having any effect on search engine-referred traffic.