Page MenuHomePhabricator

Add a "reverse" suggestion field to workaround the prefix length limitation (typos suggestion)
Closed, ResolvedPublic

Description

Today we use a suggestion field that is built against the title and redirect data. This field is used by the phrase suggester with a prefix length of 2. In other words if the typo appears in the first two characters all suggestions generated by the phrase suggester would be bad. Reducing the prefix length can hurt performances, another technique is to build a second suggestion field with a "reverse" filter.

Event Timeline

dcausse created this task.Jul 27 2015, 7:46 AM
dcausse raised the priority of this task from to Needs Triage.
dcausse updated the task description. (Show Details)
dcausse added a subscriber: dcausse.
Restricted Application added a project: Discovery. · View Herald TranscriptJul 27 2015, 7:46 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
dcausse claimed this task.Jul 28 2015, 10:03 AM
dcausse set Security to None.

Change 227478 had a related patch set uploaded (by DCausse):
WIP: Add reverse suggestion field

https://gerrit.wikimedia.org/r/227478

dcausse changed the task status from Open to Stalled.Jul 30 2015, 2:33 PM

We need to test if it's worth a try.

There's an issue in PhraseSuggester: https://github.com/elastic/elasticsearch/pull/12670

I will update the patch in cirrus to workaround the bug.

Looks like there's no workaround for this bug :(
moving to backlog until we have elasticsearch 1.7.2

Restricted Application added a subscriber: StudiesWorld. · View Herald TranscriptDec 31 2015, 5:02 AM
Deskana triaged this task as Normal priority.Dec 31 2015, 5:02 AM
Deskana added a subscriber: Deskana.
dcausse added a comment.EditedFeb 23 2016, 9:58 AM

We will have elastic 1.7.5 soon in production, I think it'd be nice to finish what we started on this task.
The top query for the last week of January 2016 was Terry Wigan which is certainly a typo to Terry Wogan (a radio and TV broadcaster who died this week).
I think that the reverse field would have helped in this case.

Restricted Application added a project: Discovery-Search. · View Herald TranscriptAug 4 2016, 4:15 PM
debt changed the task status from Stalled to Open.Aug 4 2016, 5:04 PM
debt removed a project: Discovery-Search.

Change 227478 merged by jenkins-bot:
Increase phrase suggester recall

https://gerrit.wikimedia.org/r/227478

debt closed this task as Resolved.Aug 26 2016, 4:56 PM