Interleaved results A/B test: turn off test
Closed, ResolvedPublic

Description

Let's use this ticket to turn off the A/B 2-way test for interleaved results. Expected time to have this test run is 7 days.

debt created this task.Jul 20 2017, 7:53 PM
Restricted Application edited projects, added Discovery-Search; removed Discovery-Search (Current work). · View Herald TranscriptJul 20 2017, 7:53 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
debt moved this task from Needs triage to Up Next on the Discovery-Search board.Jul 27 2017, 5:14 PM

Change 372153 had a related patch set uploaded (by EBernhardson; owner: EBernhardson):
[mediawiki/extensions/WikimediaEvents@master] Disable cirrus MLR ab test

https://gerrit.wikimedia.org/r/372153

Change 372169 had a related patch set uploaded (by Thcipriani; owner: EBernhardson):
[mediawiki/extensions/WikimediaEvents@wmf/1.30.0-wmf.13] Disable cirrus MLR ab test

https://gerrit.wikimedia.org/r/372169

Change 372170 had a related patch set uploaded (by Thcipriani; owner: EBernhardson):
[mediawiki/extensions/WikimediaEvents@wmf/1.30.0-wmf.14] Disable cirrus MLR ab test

https://gerrit.wikimedia.org/r/372170

Change 372169 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@wmf/1.30.0-wmf.13] Disable cirrus MLR ab test

https://gerrit.wikimedia.org/r/372169

Change 372170 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@wmf/1.30.0-wmf.14] Disable cirrus MLR ab test

https://gerrit.wikimedia.org/r/372170

Mentioned in SAL (#wikimedia-operations) [2017-08-16T18:29:01Z] <thcipriani@tin> Synchronized php-1.30.0-wmf.14/extensions/WikimediaEvents/modules/ext.wikimediaEvents.searchSatisfaction.js: SWAT: [[gerrit:372170|Disable cirrus MLR ab test]] T171214 (duration: 00m 51s)

Mentioned in SAL (#wikimedia-operations) [2017-08-16T18:30:32Z] <thcipriani@tin> Synchronized php-1.30.0-wmf.13/extensions/WikimediaEvents/modules/ext.wikimediaEvents.searchSatisfaction.js: SWAT: [[gerrit:372169|Disable cirrus MLR ab test]] T171214 (duration: 00m 50s)

debt closed this task as Resolved.Aug 17 2017, 6:18 PM
debt claimed this task.

This was turned off on Aug 16, 2017; we are currently awaiting analysis to begin/complete in T171215.

debt removed debt as the assignee of this task.Aug 17 2017, 6:18 PM

Change 372153 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] Disable cirrus MLR ab test

https://gerrit.wikimedia.org/r/372153

EBernhardson reopened this task as Open.Aug 18 2017, 1:01 AM

Re-opening as the sampling rates were set much lower than expected. Test now starts on aug 17th and will run through aug 24. The previous run of the test has valid data to analyze, but it only collected 1k sessions per bucket. This next run of the same test should collect 7k sessions per bucket if turned off on aug 24.

debt added a comment.Aug 22 2017, 5:34 PM

This was turned back on Aug 17 and will turn back off on Thursday, Aug 24.

Change 374394 had a related patch set uploaded (by EBernhardson; owner: EBernhardson):
[mediawiki/extensions/WikimediaEvents@master] Turn off Cirrus MLR AB Test

https://gerrit.wikimedia.org/r/374394

Change 374394 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] Turn off Cirrus MLR AB Test

https://gerrit.wikimedia.org/r/374394

Change 374432 had a related patch set uploaded (by EBernhardson; owner: EBernhardson):
[mediawiki/extensions/WikimediaEvents@wmf/1.30.0-wmf.15] Turn off Cirrus MLR AB Test

https://gerrit.wikimedia.org/r/374432

Change 374432 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@wmf/1.30.0-wmf.15] Turn off Cirrus MLR AB Test

https://gerrit.wikimedia.org/r/374432

Mentioned in SAL (#wikimedia-operations) [2017-08-28T23:13:34Z] <ebernhardson@tin> Synchronized php-1.30.0-wmf.15/extensions/WikimediaEvents/: Turn off two cirrus AB tests T171742 T171214 (duration: 00m 44s)

debt closed this task as Resolved.Aug 30 2017, 8:20 PM
debt claimed this task.