Page MenuHomePhabricator

Bucket search satisfaction analytics by query type
Open, Needs TriagePublic

Description

The new search satisfaction schema currently is very high level, it doesn't have any way to differentiate types of queries that are being served.

To allow us to measure the effectiveness of different kinds of queries and changes to those queries the backend should report on the type of query it ran and that should be included in the search schema.

This was prompted by ideas about how to deal with T103989. There are some changes we could make, but we don't know if they will be any good and need a way to measure it.

Event Timeline

EBernhardson raised the priority of this task from to Needs Triage.
EBernhardson updated the task description. (Show Details)
EBernhardson added a project: CirrusSearch.
EBernhardson added a subscriber: EBernhardson.
Restricted Application added a project: Discovery. · View Herald TranscriptJul 2 2015, 2:48 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

@Ironholds if we also change the sampling ratio based on the query type, to collect more information about a query that we are either plan to change or have already changed, how much work will that be to take into account when reporting the search satisfaction data?

Varying the sampling ratio may be unnecessary, initially we should stick to the 1 in 1000 sampling and see what results we get. But it might be that some of the queries we change might not collect enough data that way.