Let's use this ticket to turn off the A/B 2-way test for interleaved results. Expected time to have this test run is 7 days.
|Open||None||T174064 [FY 2017-18 Objective] Implement advanced search methodologies|
|Resolved||EBernhardson||T161632 [Epic] Improve search by researching and deploying machine learning to re-rank search results|
|Resolved||EBernhardson||T162369 Evaluate rescore windows for learning to rank|
|Resolved||None||T174066 [Q1 2017-18 Objective] Perform load and A/B tests on new models (interleaved search results)|
|Resolved||EBernhardson||T150032 Add support for interleaved results in 2-way A/B test|
|Resolved||debt||T171212 Interleaved results A/B test: turn on|
|Resolved||EBernhardson||T171213 Interleaved results A/B test: check that data is flowing the way we expect|
|Resolved||debt||T171214 Interleaved results A/B test: turn off test|
|Resolved||mpopov||T171215 Interleaved results A/B test: analysis of data|
|Declined||EBernhardson||T171984 Turn on test of LTR with standard AB buckets and an interleaved bucket.|
Re-opening as the sampling rates were set much lower than expected. Test now starts on aug 17th and will run through aug 24. The previous run of the test has valid data to analyze, but it only collected 1k sessions per bucket. This next run of the same test should collect 7k sessions per bucket if turned off on aug 24.