Page MenuHomePhabricator

Add option to set limit in Global Search
Open, Needs TriagePublicFeature

Description

Limiting the number of search results would improve performance, especially for queries you know or expect to be long-running.

Event Timeline

I'm not certain this is true. I seem to recall that in the case of Cloud Elastic, the backend essentially gets all the results. The only reason we limit it to 5,000 results is because the HTML payload is so big.

I see we are setting an explicit size when making requests to Cloud Elastic, so adding an input field for this should be very easy. @EBernhardson, can you confirm whether lowering the size will help performance or not?

In the general case in elastic 6.x the main performance difference with a lower limit will be time spent primarily in the highlighter, matching up positions to decide which snippet to return. This query though is a bit different, in that it is only a filter with no scoring component. We can apply "sort": ["_doc"] (might be implicit when query is filter only, not sure) which will tell elasticsearch shards to return the first N docs that match, rather than pushing into a priority queue and returning a top-n. So in theory, returning less docs might require scanning less of the indices because it can early-exit individual shard queries, and the highlighter wont have to go through and lookup positions on all those documents. On the other hand, while I haven't measured it I suspect the runtime of Global Search to be dominated by it having to issue the query > 2k times across different shards.

If it's not too painful it seems reasonable to test.