Elasticsearch 7.x has a specific field type of search-as-you-type intended to optimize autocomplete style searches where the user is typing and the application shows possible completions. That would however be blocked on an Elasticsearch cluster upgrade.

@bd808 Is this the responsibility of the search team? I can inquire whether this is on their roadmap or not

In T295942#7524220, @sdkim wrote:

@bd808 Is this the responsibility of the search team? I can inquire whether this is on their roadmap or not

No, it is our responsibility. Using any ES 7.x features would be blocked on the Search team upgrading the Elasticsearch servers that host our index, but the configuration and content of the index is managed by us.

Sorry, yes I should have specified upgrading the ES servers.

@MPhamWMF Can I ask if you all have any plans in the near-term to upgrade to ES 7.x?

@sdkim , we are planning to upgrade to ES 7.10 starting Q3, with the current estimate to be up to ~2 quarters of work.

In T295942#7527971, @sdkim wrote:

Calling out dependency on https://phabricator.wikimedia.org/T263142 for this task

That is a dependency for an implementation based on the upstream search-as-you-type feature, but not for generally addressing the current implementation deficiencies. There are a number of currently available field analysis and query type changes which could be made.

There are a number of currently available field analysis and query type changes which could be made.

@bd808 Please enumerate the changes and the outcome they would provide.

If search-as-you-type is separate from what this task was intended for I can create a separate task for it if that works for you?

In T295942#7534886, @sdkim wrote:

There are a number of currently available field analysis and query type changes which could be made.

@bd808 Please enumerate the changes and the outcome they would provide.

In T271386#7581005, @bd808 wrote:

In T271386#7580624, @Raymond_Ndibe wrote:

• Can you elaborate more on this? I'm interested in understanding your point. @bd808 can you weigh in on this too? right now we are approaching this problem through stemming but maybe we can do more?

I think that T295942: Add optimized Elasticsearch field mappings for autocomplete searches is basically talking about the same issue. There is a fancy new "search-as-you-type" feature for this in Elasticsearch 7.x which we do not currently have access to. The new feature however is really just some nice syntactic sugar for creating a custom field type that uses multiple N-gram tokenizers (one with shingle size 2 and another with shingle size 3). N-gram searches end up working a bit like a double wildcard search (*something*) but are more runtime performant at a cost of larger indexes.

Change 761691 had a related patch set uploaded (by Raymond Ndibe; author: Raymond Ndibe):

[wikimedia/toolhub@main] Search: Add optimized Elasticsearch field mappings for autocomplete searches

https://gerrit.wikimedia.org/r/761691

Change 761691 merged by jenkins-bot:

[wikimedia/toolhub@main] Search: Add optimized field mappings for autocomplete searches

https://gerrit.wikimedia.org/r/761691

Change 770638 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[operations/deployment-charts@master] toolhub: Bump container version to 2022-03-15-002555-production

https://gerrit.wikimedia.org/r/770638

Change 770638 merged by jenkins-bot:

[operations/deployment-charts@master] toolhub: Bump container version to 2022-03-15-002555-production

https://gerrit.wikimedia.org/r/770638