We need to be able to set a sampling rate and then bucket users within the sample to perform A/B tests. This should only measure the *second* query. If the users initial search returned results they will not be part of these tests. I would like to measure the effectiveness of different secondary queries we can run to ensure something without results ends up having results. Thus this needs to happen on the backend when performing a search.
We are initially measuring the zero results rate, so this should tie into a fairly straight forward EventLogging schema that records some sort of test identifier, what bucket the search was put into, and the number of results that were returned to the user.
It might also be nice if this could run multiple tests at the same time. Setting up individual tests is outside the scope of this ticket, this is just about getting everything in place and having a schema to log it to.