The downstream changes were put in place because of long running global fulltext searches.
The Global Search (in my limited understanding) queries all typeahead sources, and only one typeahead source is Projects.

So I'd say putting a custom patch just for Projects is not the best approach.

In upstream rPHAB61ce56ff I restricted the number of search tokens to 61. This is deployed on our Wikimedia instance.
If global fulltext search was the reason for this code, then a one-line change to lower the return value of PhutilSearchQueryCompiler.php::getMaxQueryTokens() makes more sense.
That might be alright now that we moved e.g. Phatility search from the Global Search to the Maniphest Search in T398305: Query Maniphest Advanced Search instead of Phabricator Primary Global Search.
However, I have no clue which other external stuff may query our global search with lots of search tokens.

In any case, the current downstream code which does if (count($filtered) > 5) makes no sense when not in a global fulltext search context but in a project typeahead context, as the typeahead search interprets this as project.name LIKE 'fooo barr kdlf lkdj wejh ktjg bleh%') in SQL. No tokenization.

And the current if (strlen($token) > 3) { $filtered[] = $token; } leading to no tokens at all, ending up with return $this; as-was-before, is a problem.
FYI if we really wanted to properly drop query terms based on short length, then the right place to do that is in PhutilSearchQueryCompiler::tokenizeQuery() (but do note the use of phutil_utf8_is_cjk(), plus would have to take into account the substring operator ~ here).

So in summary, I think we should

• either reduce or remove custom if (strlen($token) > 3) { $filtered[] = $token; } to mitigate the problem of useless search results (e.g. to only potentially appear with single letter typeahead input which may still trash fulltext search for a single-letter though which is acceptable),
  • or introduce code in PhutilSearchQueryCompiler to drop very short tokens
• potentially remove the custom if (count($filtered) > 5) condition which for no reason special-cases only project search (though it does work - e.g. when entering seven tokens in the "Name" field on /project/query/advanced/ the last ones are ignored as the SQL query shows)
• likely lower the general threshold for the number of search tokens in freetext search in PhutilSearchQueryCompiler::getMaxQueryTokens() from 61 to something arbitrary like 16 (Logstash will tell us how often searches will break with an error).

After doing some changes, I'd like to revisit T360701 which may not be fully fixable as we may have >102 #Wikibase# projects so search result pagination interferes, as kinda described by Evan in https://secure.phabricator.com/T8510#180681.

Current behavior is both sad and funny:

I considered allowing a different number of search tokens depending on whether the user is logged in, but it's impossible to reliably set/pass $this->getViewer() in all PhutilSearchQueryCompiler constructors to PhutilSearchQueryCompiler::getMaxQueryTokens() according to my local testing.

aklapper opened https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/105

Adjust numerous custom search query length restrictions

aklapper merged https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/105

Adjust numerous custom search query length restrictions