I've pulled two sets of logs from fluorine to run through the new suggestions api. Based on this data it looks like we might be able to achieve almost a 40% reduction in the zero result rate of prefix searches by replacing it with the completion suggester.

First set:

This is an unsampled log and contains all prefix searches for enwiki_content that returned no results between 2015-08-26 08:02:23 and 2015-08-26 08:15:49.

API	results	no results	zero %
prefix_search	0	100000	100%
completion_suggest	38295	61731	62%

Second set

This is every 100th(take 1, throw away 99, repeat) prefix search issued to enwiki_content between 2015-08-25 08:03:30 and 2015-08-26 08:02:23

API	results	no results	zero %
prefix_search	288210	134469	32%
completion_suggest	340562	82183	19%

• ksmith added a project: OKR-Work.Aug 26 2015, 11:03 PM

EBernhardson moved this task from not in use - please delete to Needs review on the Discovery-Search (Current work) board.Aug 26 2015, 11:22 PM

Ran same test for dewiki

First set

Unsampled log contains all prefix searches for dewiki_content that returned no results between 2015-08-26 08:02:23 to 2015-08-26 08:48:07

API	results	no results	zero %
prefix_search	0	100000	100%
completion_suggest	44678	55333	55%

Second set

every 100th(take 1, throw away 99, repeat) prefix search issued to dewiki_content between 2015-08-25 08:03:31 to 2015-08-26 08:02:23

API	results	no results	zero %
prefix_search	58641	23283	33%
completion_suggest	69709	12216	15%

Sweet! Nice report.

• Deskana moved this task from Needs review to Needs Reporting on the Discovery-Search (Current work) board.Aug 27 2015, 4:33 PM

• Deskana mentioned this in T108766: [epic] Setup elasticsearch cluster in labs and build out necessary tooling to use it for hypothesis testing against production data.Aug 31 2015, 3:25 AM

• Deskana mentioned this in T108767: Make available an XL labs instance with ~350GB available disk space. .

• Deskana mentioned this in T111078: Run A/B test on the search suggester to measure zero results rate, starting on 2015-09-08.Sep 1 2015, 6:37 PM

• Deskana mentioned this in T105746: Build out an API that exposes ElasticSearch suggester results for a given query.Sep 2 2015, 12:31 AM

• Deskana mentioned this in T111139: Provide documentation for cirrus-suggest API.Sep 2 2015, 3:15 AM

• Deskana moved this task from Needs Reporting to Resolved on the Discovery-Search (Current work) board.Sep 9 2015, 2:24 AM

Run tests to measure the expected zero results change of changing suggestions algorithmClosed, ResolvedPublicActions

Related Objects

Event Timeline

First set:

Second set

First set

Second set

Run tests to measure the expected zero results change of changing suggestions algorithm
Closed, ResolvedPublic
Actions