This will probably be triggered by bucketing users in the satisfaction schema code, along with a backend trigger configured from mediawiki-config for the CirrusSearch\UserTesting to setup the appropriate variables.
Description
Details
Status | Subtype | Assigned | Task | ||
---|---|---|---|---|---|
Resolved | • Deskana | T143585 Initial BM25 A/B Test | |||
Resolved | mpopov | T143587 Verify data pipeline for bm25 AB test | |||
Resolved | debt | T143586 Turn on BM25 test | |||
Resolved | EBernhardson | T143590 Reindex codfw search cluster for the bm25 AB test |
Event Timeline
Change 306721 had a related patch set uploaded (by DCausse):
CirrusSearch BM25 A/B test config
Mentioned in SAL [2016-08-29T18:23:20Z] <thcipriani@tin> Synchronized wmf-config/CirrusSearch-common.php: SWAT: [[gerrit:306721|CirrusSearch BM25 A/B test config (T143586)]] (duration: 00m 46s)
Mentioned in SAL [2016-08-29T18:29:09Z] <thcipriani@tin> Synchronized wmf-config/CirrusSearch-common.php: REVERT SWAT: [[gerrit:306721|CirrusSearch BM25 A/B test config (T143586)]] (duration: 00m 48s)
Change 307535 had a related patch set uploaded (by EBernhardson):
Turn on CirrusSearch bm25 A/B test
Change 307536 had a related patch set uploaded (by EBernhardson):
Turn on CirrusSearch bm25 A/B test
While testing before deployment we have found that the index settings created in prod do not match those used for tuning in relforge. This results in some poorly performing results, such as 'the query 'kennedy' moving the page 'John F. Kennedy' from the 3rd position to the 11th position. Will need to reindex one more time with the appropriate settings before running the test.
I updated the name of this ticket to remove the date. We hope to have this test starting soon.