Maniphest T420427

Search shouldn't trim trailing space when suggesting suggestions
Closed, ResolvedPublic3 Estimated Story PointsBUG REPORT
Actions

Assigned To

Authored By

	Jack_who_built_the_house
	Mar 18 2026, 4:15 AM

Project Tags

Referenced Files

	F72997282: image.png
	Mar 18 2026, 4:19 AM

	F72996426: image.png
	Mar 18 2026, 4:15 AM

	F72996387: image.png
	Mar 18 2026, 4:15 AM

	F72996921: image.png
	Mar 18 2026, 4:15 AM

	F72996027: image.png
	Mar 18 2026, 4:15 AM

Subscribers

Aklapper

Bugreporter

Jack_who_built_the_house

Lucas_Werkmeister_WMDE

Description

Steps to replicate the issue:

You want to get an article about Go, programming language. Or Go, game. Or anything where "Go" is used as a word, not prefix.
You open https://en.wikipedia.org/wiki/Main_Page.
You click the search input.
You type "Go ". Since you typed that space, you definitely won't see suggestions like "Google". Right? Right?

What happens?

What should have happened instead?
If the query ends with a space, you shouldn't get results with the query as a prefix.

Thoughts

This is expecially bad in Wiktionary where you there are a lot of word–prefix collisions. Want to quickly see entries that start with the word word? You're out of luck.

Google handles this well.

Details

Related Changes in Gerrit:

Subject	Author	Repo	Branch	Lines +/-
mediawiki.skinning.typeaheadSearch: keep trailing spaces	DCausse	mediawiki/core	master	+27 -1
cirrus: use a keywork tokenizer for the plain field for autocomplete	DCausse	operations/mediawiki-config	master	+5 -25
search: add alt. completion indices to test keyword tokenizer (2/2)	DCausse	operations/mediawiki-config	master	+2 -2
search: fix alt. completion indices to test keyword tokenizer	DCausse	operations/mediawiki-config	master	+20 -21
search: add alt. completion indices to test keyword tokenizer (1/2)	DCausse	operations/mediawiki-config	master	+24 -1
search: add space-discount for wikidata custom prefix search profiles	DCausse	operations/mediawiki-config	master	+4 -0
Allow setting the completion tokenizer on the plain field	DCausse	mediawiki/extensions/CirrusSearch	master	+9 -2

Customize query in gerrit

Related Objects

Mentioned In: rMSFEc673382915c1: rustversion: Reach OTel parity with the JS orchestrator's tracer.js
T425018: Enable Latin-to-Devanagari Transliteration second-try search on Hindi Wikis
T425433: Wrong reuse message when creating a new reference
T417648: [MEX] M4 - improve findability of properties on lookups
Mentioned Here: T425018: Enable Latin-to-Devanagari Transliteration second-try search on Hindi Wikis
T425433: Wrong reuse message when creating a new reference
rCICF12830376bd04: Update patch set 2
T417648: [MEX] M4 - improve findability of properties on lookups

Event Timeline

Jack_who_built_the_house created this task.Mar 18 2026, 4:15 AM

Restricted Application added a project: Discovery-Search. · View Herald TranscriptMar 18 2026, 4:15 AM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Jack_who_built_the_house updated the task description. (Show Details)Mar 18 2026, 4:16 AM

Jack_who_built_the_house updated the task description. (Show Details)

Jack_who_built_the_house updated the task description. (Show Details)Mar 18 2026, 4:19 AM

Jack_who_built_the_house updated the task description. (Show Details)Mar 18 2026, 4:29 AM

The search term should still be left trimmed. We should only remove right trim.

Similar issue for Wikidata search: T417648: [MEX] M4 - improve findability of properties on lookups

Lucas_Werkmeister_WMDE mentioned this in T417648: [MEX] M4 - improve findability of properties on lookups.Mar 18 2026, 9:36 AM

In T420427#11721930, @Bugreporter wrote:

The search term should still be left trimmed. We should only remove right trim.

Right.

Jack_who_built_the_house renamed this task from Search shouldn't trim space when suggesting suggestions to Search shouldn't trim trailing space when suggesting suggestions.Mar 18 2026, 9:55 AM

Jack_who_built_the_house updated the task description. (Show Details)

EBernhardson moved this task from needs triage to 2026.03.03 - 2026.04.03 on the Discovery-Search board.Mar 23 2026, 4:13 PM

EBernhardson edited projects, added: Discovery-Search (2026.03.03 - 2026.04.03); removed: Discovery-Search.

EBernhardson set the point value for this task to 3.Mar 23 2026, 4:19 PM

EBernhardson moved this task from Incoming to Ready for Dev on the Discovery-Search (2026.03.03 - 2026.04.03) board.

dcausse claimed this task.Apr 2 2026, 12:49 PM

dcausse moved this task from Ready for Dev to In Progress on the Discovery-Search (2026.03.03 - 2026.04.03) board.

The trailing spaces are not trimmed by MW but because the completion algorithm is tokenizing the query and the titles spaces are ignored most of the time.
We can attempt to change the behavior but I'd suggest to run a quick test to see possible undesirable side-effects.
There is also an an issue with wikibase prefix search (which is not using the same algorithm) where trailing spaces are supposed to influence scoring but this got disabled by some custom scoring profiles.
We can fix custom profiles (en, de, es fr) to let trailing spaces influence the ranking but I think this is different from what is requested in T417648.
In T417648 I understand that the trailing space should be used as a signal to tell the search engine to look for "exact" matches on the trimmed search string. The wikibase prefix search should already have a ranking signal on an "exact" label/alias match from the trimmed search string but I suspect the different weights are not doing what we expect in this case.

Change #1267123 had a related patch set uploaded (by DCausse; author: DCausse):

[mediawiki/extensions/CirrusSearch@master] Allow setting the completion tokenizer on the plain field

https://gerrit.wikimedia.org/r/1267123

gerritbot added a project: Patch-For-Review.Apr 2 2026, 4:48 PM

Change #1267130 had a related patch set uploaded (by DCausse; author: DCausse):

[operations/mediawiki-config@master] search: add space-discount for wikidata custom prefix search profiles

https://gerrit.wikimedia.org/r/1267130

Change #1267123 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] Allow setting the completion tokenizer on the plain field

https://gerrit.wikimedia.org/r/1267123

ReleaseTaggerBot added a project: MW-1.46-notes (1.46.0-wmf.23; 2026-04-07).Apr 3 2026, 5:00 PM

Change #1269464 had a related patch set uploaded (by DCausse; author: DCausse):

[operations/mediawiki-config@master] search: add alt. completion indices to test keyword tokenizer (1/2)

https://gerrit.wikimedia.org/r/1269464

Change #1269465 had a related patch set uploaded (by DCausse; author: DCausse):

[operations/mediawiki-config@master] search: add alt. completion indices to test keyword tokenizer (2/2)

https://gerrit.wikimedia.org/r/1269465

pfischer edited projects, added: Discovery-Search (2026.04.06 - 2026.05.01); removed: Discovery-Search (2026.03.03 - 2026.04.03).Apr 13 2026, 1:57 PM

pfischer moved this task from Incoming to In Progress on the Discovery-Search (2026.04.06 - 2026.05.01) board.

Change #1267130 abandoned by DCausse:

[operations/mediawiki-config@master] search: add space-discount for wikidata custom prefix search profiles

Reason:

not needed if I908966a32fc264c5084719337812ca303bac509c is shipped

https://gerrit.wikimedia.org/r/1267130

Change #1269464 merged by jenkins-bot:

[operations/mediawiki-config@master] search: add alt. completion indices to test keyword tokenizer (1/2)

https://gerrit.wikimedia.org/r/1269464

Mentioned in SAL (#wikimedia-operations) [2026-04-29T07:17:19Z] <dcausse@deploy1003> Started scap sync-world: Backport for [[gerrit:1278509|Completion: fix near match field name (T417648)]], [[gerrit:1269464|search: add alt. completion indices to test keyword tokenizer (1/2) (T420427)]]

Mentioned in SAL (#wikimedia-operations) [2026-04-29T07:19:18Z] <dcausse@deploy1003> dcausse: Backport for [[gerrit:1278509|Completion: fix near match field name (T417648)]], [[gerrit:1269464|search: add alt. completion indices to test keyword tokenizer (1/2) (T420427)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2026-04-29T07:24:26Z] <dcausse@deploy1003> Finished scap sync-world: Backport for [[gerrit:1278509|Completion: fix near match field name (T417648)]], [[gerrit:1269464|search: add alt. completion indices to test keyword tokenizer (1/2) (T420427)]] (duration: 07m 07s)

Change #1283037 had a related patch set uploaded (by DCausse; author: DCausse):

[operations/mediawiki-config@master] search: fix alt. completion indices to test keyword tokenizer

https://gerrit.wikimedia.org/r/1283037

Change #1283037 merged by jenkins-bot:

[operations/mediawiki-config@master] search: fix alt. completion indices to test keyword tokenizer

https://gerrit.wikimedia.org/r/1283037

Mentioned in SAL (#wikimedia-operations) [2026-05-06T07:31:24Z] <awight@deploy1003> Started scap sync-world: Backport for [[gerrit:1283101|VE: Avoid counting all refs when listIndex is undefined (T425433)]], [[gerrit:rCICF12830376bd04|search: fix alt. completion indices to test keyword tokenizer (T420427)]], [[gerrit:1283041|search: enable Latin-to-Devanagari transliteration second-chance (T425018)]]

Mentioned in SAL (#wikimedia-operations) [2026-05-06T07:33:20Z] <awight@deploy1003> wmde-fisch, awight, dcausse: Backport for [[gerrit:1283101|VE: Avoid counting all refs when listIndex is undefined (T425433)]], [[gerrit:rCICF12830376bd04|search: fix alt. completion indices to test keyword tokenizer (T420427)]], [[gerrit:1283041|search: enable Latin-to-Devanagari transliteration second-chance (T425018)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can

Mentioned in SAL (#wikimedia-operations) [2026-05-06T07:40:23Z] <awight@deploy1003> Finished scap sync-world: Backport for [[gerrit:1283101|VE: Avoid counting all refs when listIndex is undefined (T425433)]], [[gerrit:rCICF12830376bd04|search: fix alt. completion indices to test keyword tokenizer (T420427)]], [[gerrit:1283041|search: enable Latin-to-Devanagari transliteration second-chance (T425018)]] (duration: 08m 58s)

Change #1269465 merged by jenkins-bot:

[operations/mediawiki-config@master] search: add alt. completion indices to test keyword tokenizer (2/2)

https://gerrit.wikimedia.org/r/1269465

Mentioned in SAL (#wikimedia-operations) [2026-05-07T07:44:20Z] <dcausse@deploy1003> Started scap sync-world: Backport for [[gerrit:1269465|search: add alt. completion indices to test keyword tokenizer (2/2) (T420427)]]

Mentioned in SAL (#wikimedia-operations) [2026-05-07T07:46:22Z] <dcausse@deploy1003> dcausse: Backport for [[gerrit:1269465|search: add alt. completion indices to test keyword tokenizer (2/2) (T420427)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2026-05-07T07:54:06Z] <dcausse@deploy1003> Finished scap sync-world: Backport for [[gerrit:1269465|search: add alt. completion indices to test keyword tokenizer (2/2) (T420427)]] (duration: 09m 46s)

Maintenance_bot removed a project: Patch-For-Review.May 7 2026, 8:31 AM

Tested on enwiki and enwiktionary, I don't see any obvious nasty side-effects using a test index (example results https://en.wikipedia.org/w/api.php?action=opensearch&search=go+&cirrusCompletionAltIndexId=1).
Index sizes are equivalent:
eqiad:

green open enwiki_titlesuggest_1778121002         5ex7e_VhSvmQGbp6WsJZsA  4 3  10634814       60   20.4gb    8.1gb
green open enwiki_titlesuggest_alt_1_first        eW_5iR1dRAW3C1aIMdmQfA  4 3  10634814       60   20.4gb    8.1gb
green open enwiktionary_titlesuggest_alt_1_first  qEOBcdIzQZ6dAa5Ed2e-QA  2 2   8837105        0    5.7gb    2.8gb
green open enwiktionary_titlesuggest_1778121692   whajQZggRBClFe6BbNOsJw  2 2   8837105        0    5.7gb    2.8gb

codfw:

green open enwiki_titlesuggest_1778121003         qPa5in8SQ3CcyPPxm1iqDQ  4 3  10634815       60   19.3gb    7.1gb
green open enwiki_titlesuggest_alt_1_first        1Yb13fIlQH-u2758x6VlKA  4 3  10634815       60   20.4gb    8.1gb
green open enwiktionary_titlesuggest_alt_1_first  7sZ_0upiTdGLtVmYzOhz5w  2 2   8837108        0    5.7gb    2.8gb
green open enwiktionary_titlesuggest_1778121724   89YUJOgAQuemLi9UrfwdfA  2 2   8837108        0      5gb    2.1gb

Change #1284628 had a related patch set uploaded (by DCausse; author: DCausse):

[operations/mediawiki-config@master] cirrus: use a keywork tokenizer for the plain field for autocomplete

https://gerrit.wikimedia.org/r/1284628

gerritbot added a project: Patch-For-Review.May 7 2026, 12:37 PM

dcausse moved this task from In Progress to Needs Review on the Discovery-Search (2026.04.06 - 2026.05.01) board.May 7 2026, 12:38 PM

Change #1284628 merged by jenkins-bot:

[operations/mediawiki-config@master] cirrus: use a keywork tokenizer for the plain field for autocomplete

https://gerrit.wikimedia.org/r/1284628

Maintenance_bot removed a project: Patch-For-Review.May 12 2026, 7:30 AM

jforrester opened https://gitlab.wikimedia.org/repos/abstract-wiki/wikifunctions/function-evaluator/-/merge_requests/531

rustversion: Reach OTel parity with the JS orchestrator's tracer.js

Jdforrester-WMF mentioned this in rMSFEc673382915c1: rustversion: Reach OTel parity with the JS orchestrator's tracer.js.May 13 2026, 11:14 PM

jforrester merged https://gitlab.wikimedia.org/repos/abstract-wiki/wikifunctions/function-evaluator/-/merge_requests/531

rustversion: Reach OTel parity with the JS orchestrator's tracer.js

Maintenance_bot removed a project: Patch-For-Review.May 14 2026, 9:31 PM

This is has been deployed and appears to work on the backend but not from the main search widget (works as expected from Special:Search autocomplete), I suspect that some trimming might be happening on the UI, looking into it.

Change #1288455 had a related patch set uploaded (by DCausse; author: DCausse):

[mediawiki/core@master] mediawiki.skinning.typeaheadSearch: keep trailing spaces

https://gerrit.wikimedia.org/r/1288455

gerritbot added a project: Patch-For-Review.May 18 2026, 7:47 AM

dcausse moved this task from In Progress to Needs Review on the Discovery-Search (2026.04.06 - 2026.05.01) board.May 18 2026, 7:47 AM

pfischer edited projects, added: Discovery-Search (2026.05.04 - 2026.05.29); removed: Discovery-Search (2026.04.06 - 2026.05.01).May 18 2026, 1:40 PM

pfischer moved this task from Incoming to Needs Review on the Discovery-Search (2026.05.04 - 2026.05.29) board.

Change #1288455 merged by jenkins-bot:

[mediawiki/core@master] mediawiki.skinning.typeaheadSearch: keep trailing spaces

https://gerrit.wikimedia.org/r/1288455

ReleaseTaggerBot added a project: MW-1.47-notes (1.47.0-wmf.4; 2026-05-26).May 22 2026, 12:00 PM

Maintenance_bot removed a project: Patch-For-Review.May 22 2026, 12:31 PM