The completion suggester has several options that are evaluated at index time to build its index.
This is particularly challenging for A/B testing since we build and query a single index at any given time.
We could add the ability to create multiple completion indices for the same wiki to make A/B testing easier.
The details to achieve this are still unclear but ideally a single run of UpdateSuggesterIndex should be necessary.
It might be possible to inspect wgCirrusSearchUserTesting and determine if an extra index is necessary or not.
Cleaning up the extra indices might be challenging once the test is done and we should ponder how to achieve this:
- doing the cleanup manually
- have automated that identifies old & unused completion that could match some name conventions used for A/B tests
- have a two steps un-deploy procedure with an intermediate step to instruct UpdateSuggesterIndex to delete the extra index
AC:
- Provide the ability to create extra completion indices based on wgCirrusSearchUserTesting
- The extra indices should ideally be built during the same UpdateSuggesterIndex execution and re-use the same loop to extract the docs from the main indices
- Create a ticket to A/B test an index time feature like using defaultsort on a set of wikis