turn off sistersearch A/B/C test on 2017-02-21
Closed, ResolvedPublic

Description

This ticket is a reminder to turn off the test the week of Feb 13, 2017 - as long as we've received sufficient data.

debt created this task.Feb 13 2017, 9:21 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 13 2017, 9:21 AM
Aklapper renamed this task from turn off test to turn off test in week of 2017-02-13.Feb 13 2017, 1:31 PM
Deskana renamed this task from turn off test in week of 2017-02-13 to turn off sistersearch A/B/C test in week of 2017-02-13.Feb 13 2017, 5:50 PM
Deskana renamed this task from turn off sistersearch A/B/C test in week of 2017-02-13 to turn off sistersearch A/B/C test on 2017-02-21.Feb 14 2017, 6:07 PM
Deskana added a subscriber: Deskana.

Now updated to 21st February, as sampling rate was too low and it's being increased today. Running tests for a week to capture some degree of periodicity is good.

debt added a comment.Feb 14 2017, 6:28 PM

Reposting from this comment for clarity in updating the sampling (bucketing) rate and why the test is being extended:

After discussing in IRC decided to increase from 1:200 EL, 1:10 test (1:2000 overall to test) on two low volume wikis (catalan and persian) to 1:10 EL, 1:2 test, for 1:20 overall (100x more sessions). The remaining two medium volume wikis (italian and polish) increased to 1:50 EL, 1:2 test, for 1:100 overall (20x more sessions).

Change 339008 had a related patch set uploaded (by EBernhardson):
Turn off sister search AB test.

https://gerrit.wikimedia.org/r/339008

Change 339008 merged by jenkins-bot:
Turn off sister search AB test.

https://gerrit.wikimedia.org/r/339008

Change 339010 had a related patch set uploaded (by EBernhardson):
Turn off sister search AB test.

https://gerrit.wikimedia.org/r/339010

Change 339011 had a related patch set uploaded (by EBernhardson):
Turn off sister search AB test.

https://gerrit.wikimedia.org/r/339011

Change 339010 merged by jenkins-bot:
Turn off sister search AB test.

https://gerrit.wikimedia.org/r/339010

Change 339011 merged by jenkins-bot:
Turn off sister search AB test.

https://gerrit.wikimedia.org/r/339011

Mentioned in SAL (#wikimedia-operations) [2017-02-22T00:16:01Z] <thcipriani@tin> Synchronized php-1.29.0-wmf.13/extensions/WikimediaEvents/modules/ext.wikimediaEvents.searchSatisfaction.js: SWAT: [[gerrit:339011|Turn off sister search AB test.]] T157942 (duration: 00m 43s)

Mentioned in SAL (#wikimedia-operations) [2017-02-22T00:17:07Z] <thcipriani@tin> Synchronized php-1.29.0-wmf.12/extensions/WikimediaEvents/modules/ext.wikimediaEvents.searchSatisfaction.js: SWAT: [[gerrit:339010|Turn off sister search AB test.]] T157942 (duration: 00m 39s)