Page MenuHomePhabricator

Check in sameAs A/B test results
Closed, ResolvedPublicJan 13 2019

Description

Background

We would like to check in periodically with the sameAs A/B test data to ensure no major negative changes have occurred since the A/B test was rolled out

Acceptance criteria

  • Answer the following questions:
  • Are pageviews in the test and control groups dramatically different?
  • Are there any other reasons for concern?
  • Generate a graph of the following:
  • pageviews per day in test and control groups for enwiki
  • pageviews per day in test and control groups for all wikis except enwiki
  • pageviews per day in test and control groups for all wikis

Details

Due Date
Jan 13 2019, 11:00 PM

Event Timeline

ovasileva triaged this task as High priority.Dec 5 2018, 11:05 AM
ovasileva created this task.
Restricted Application changed the subtype of this task from "Task" to "Deadline". · View Herald TranscriptDec 5 2018, 11:05 AM
Niedzielski added a subscriber: Niedzielski.

I think T211190 should be a child of this since it's the initial checkin. I've made the change. Please revert if I'm mistaken.

mpopov moved this task from Triage to Backlog on the Product-Analytics board.Dec 6 2018, 9:19 PM
mpopov moved this task from Backlog to Next Up on the Product-Analytics board.Jan 15 2019, 7:16 PM
mpopov moved this task from Next Up to Doing on the Product-Analytics board.Jan 16 2019, 4:36 PM
mpopov closed this task as Resolved.EditedJan 16 2019, 4:40 PM

Just updated my database of sampled pages (using December 2018 snapshot) and recounted pageviews from 2018-11-01 to 2019-01-15 (code & data over at GitHub). There has not been any change, up or down since the rollout:

@ovasileva: we can probably just turn it off at this point as it's pretty clear that it's not having any effect on search engine-referred traffic.