Page MenuHomePhabricator

Add the ability to A/B test features of the Cirrus CompletionSuggester built at index time
Closed, ResolvedPublic8 Estimated Story Points

Description

The completion suggester has several options that are evaluated at index time to build its index.
This is particularly challenging for A/B testing since we build and query a single index at any given time.

We could add the ability to create multiple completion indices for the same wiki to make A/B testing easier.

The details to achieve this are still unclear but ideally a single run of UpdateSuggesterIndex should be necessary.
It might be possible to inspect wgCirrusSearchUserTesting and determine if an extra index is necessary or not.
Cleaning up the extra indices might be challenging once the test is done and we should ponder how to achieve this:

  • doing the cleanup manually
  • have automated that identifies old & unused completion that could match some name conventions used for A/B tests
  • have a two steps un-deploy procedure with an intermediate step to instruct UpdateSuggesterIndex to delete the extra index

AC:

  • Provide the ability to create extra completion indices based on wgCirrusSearchUserTesting
  • The extra indices should ideally be built during the same UpdateSuggesterIndex execution and re-use the same loop to extract the docs from the main indices
  • Create a ticket to A/B test an index time feature like using defaultsort on a set of wikis

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Gehel triaged this task as High priority.Apr 14 2025, 3:26 PM
Gehel moved this task from needs triage to Next Projects on the Discovery-Search board.
pfischer set the point value for this task to 8.Jun 23 2025, 3:31 PM

Change #1186493 had a related patch set uploaded (by DCausse; author: DCausse):

[mediawiki/extensions/Elastica@master] Do not actually sleep during unit tests

https://gerrit.wikimedia.org/r/1186493

Change #1186494 had a related patch set uploaded (by DCausse; author: DCausse):

[mediawiki/extensions/Elastica@master] Add support for "alternate" indices

https://gerrit.wikimedia.org/r/1186494

Change #1186495 had a related patch set uploaded (by DCausse; author: DCausse):

[mediawiki/extensions/CirrusSearch@master] Refactor UpdateSuggesterIndex

https://gerrit.wikimedia.org/r/1186495

Change #1187449 had a related patch set uploaded (by DCausse; author: DCausse):

[mediawiki/extensions/CirrusSearch@master] Add CirrusSearchAtlernativeIndices

https://gerrit.wikimedia.org/r/1187449

Change #1186493 merged by jenkins-bot:

[mediawiki/extensions/Elastica@master] Do not actually sleep during unit tests

https://gerrit.wikimedia.org/r/1186493

Change #1186494 merged by jenkins-bot:

[mediawiki/extensions/Elastica@master] Add support for "alternative" indices

https://gerrit.wikimedia.org/r/1186494

Change #1186495 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] Refactor UpdateSuggesterIndex

https://gerrit.wikimedia.org/r/1186495

Change #1187449 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] Add CirrusSearchAtlernativeIndices

https://gerrit.wikimedia.org/r/1187449

Change #1189158 had a related patch set uploaded (by DCausse; author: DCausse):

[mediawiki/extensions/CirrusSearch@master] Completion: only pull reasonable candidates from defaultsort

https://gerrit.wikimedia.org/r/1189158

Change #1189158 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] Completion: only pull reasonable candidates from defaultsort

https://gerrit.wikimedia.org/r/1189158