Page MenuHomePhabricator

Turn on BM25 test
Closed, ResolvedPublic

Description

This will probably be triggered by bucketing users in the satisfaction schema code, along with a backend trigger configured from mediawiki-config for the CirrusSearch\UserTesting to setup the appropriate variables.

Event Timeline

Change 306721 had a related patch set uploaded (by DCausse):
CirrusSearch BM25 A/B test config

https://gerrit.wikimedia.org/r/306721

debt triaged this task as Medium priority.Aug 25 2016, 10:14 PM

Change 306948 had a related patch set uploaded (by DCausse):
Turn on bm25 A/B test

https://gerrit.wikimedia.org/r/306948

Change 306721 merged by jenkins-bot:
CirrusSearch BM25 A/B test config

https://gerrit.wikimedia.org/r/306721

Mentioned in SAL [2016-08-29T18:23:20Z] <thcipriani@tin> Synchronized wmf-config/CirrusSearch-common.php: SWAT: [[gerrit:306721|CirrusSearch BM25 A/B test config (T143586)]] (duration: 00m 46s)

Mentioned in SAL [2016-08-29T18:29:09Z] <thcipriani@tin> Synchronized wmf-config/CirrusSearch-common.php: REVERT SWAT: [[gerrit:306721|CirrusSearch BM25 A/B test config (T143586)]] (duration: 00m 48s)

Change 306948 merged by jenkins-bot:
Turn on CirrusSearch bm25 A/B test

https://gerrit.wikimedia.org/r/306948

Change 307535 had a related patch set uploaded (by EBernhardson):
Turn on CirrusSearch bm25 A/B test

https://gerrit.wikimedia.org/r/307535

Change 307536 had a related patch set uploaded (by EBernhardson):
Turn on CirrusSearch bm25 A/B test

https://gerrit.wikimedia.org/r/307536

While testing before deployment we have found that the index settings created in prod do not match those used for tuning in relforge. This results in some poorly performing results, such as 'the query 'kennedy' moving the page 'John F. Kennedy' from the 3rd position to the 11th position. Will need to reindex one more time with the appropriate settings before running the test.

debt renamed this task from Turn on BM25 test aug 29th to Turn on BM25 test .Aug 30 2016, 8:16 PM
debt subscribed.

I updated the name of this ticket to remove the date. We hope to have this test starting soon.

Change 307536 merged by jenkins-bot:
Turn on CirrusSearch bm25 A/B test

https://gerrit.wikimedia.org/r/307536

Change 307535 merged by jenkins-bot:
Turn on CirrusSearch bm25 A/B test

https://gerrit.wikimedia.org/r/307535

debt claimed this task.