Page MenuHomePhabricator

Always provide minimum_should_match in bool queries
Closed, ResolvedPublic

Description

[2021-08-12T18:00:46,822][WARN ][o.e.d.i.q.BoolQueryBuilder] [gXPW_Qb] Should clauses in the filter context will no longer automatically set the minimum should match to 1 in the next major version. You should group them in a [filter] clause or explicitly set [minimum_should_match] to 1 to restore this behavior in the next major version.

Event Timeline

MPhamWMF moved this task from needs triage to elastic / cirrus on the Discovery-Search board.

Change 775276 had a related patch set uploaded (by EJoseph; author: EJoseph):

[mediawiki/extensions/CirrusSearch@master] Explicitly set minimum_should_match to 1

https://gerrit.wikimedia.org/r/775276

Change 775276 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] Explicitly set minimum_should_match to 1

https://gerrit.wikimedia.org/r/775276

Change 787701 had a related patch set uploaded (by DCausse; author: DCausse):

[mediawiki/extensions/CirrusSearch@master] Filters::booleanOr should set minimul_should_match to 1

https://gerrit.wikimedia.org/r/787701

Change 787705 had a related patch set uploaded (by DCausse; author: DCausse):

[mediawiki/extensions/WikibaseCirrusSearch@master] Temporarily comment one test case

https://gerrit.wikimedia.org/r/787705

Change 787706 had a related patch set uploaded (by DCausse; author: DCausse):

[mediawiki/extensions/WikibaseCirrusSearch@master] Fix and re-enable test related to minimum_should_match

https://gerrit.wikimedia.org/r/787706

Change 787523 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/extensions/WikibaseMediaInfo@master] Update minimum_should_match for deprecations in 7

https://gerrit.wikimedia.org/r/787523

Change 787705 merged by jenkins-bot:

[mediawiki/extensions/WikibaseCirrusSearch@master] Temporarily comment one test case

https://gerrit.wikimedia.org/r/787705

Change 787701 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] Filters::booleanOr should set minimul_should_match to 1

https://gerrit.wikimedia.org/r/787701

Change 787706 merged by jenkins-bot:

[mediawiki/extensions/WikibaseCirrusSearch@master] Fix and re-enable test related to minimum_should_match

https://gerrit.wikimedia.org/r/787706

Change 787523 merged by jenkins-bot:

[mediawiki/extensions/WikibaseMediaInfo@master] Update minimum_should_match for deprecations in 7

https://gerrit.wikimedia.org/r/787523

Change 792278 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/extensions/WikibaseCirrusSearch@master] haslicense: Apply minimum_should_match for elastic 7.x

https://gerrit.wikimedia.org/r/792278

Unfortunately incomplete, reviewing deprecation logs since turning them on last week shows both CirrusSearch and MediaSearch are still triggering this warning. Needs investigation.

Cirrus reproduction: https://en.wikipedia.org/w/index.php?sort=random&search=foobar+insource%3Aqwerty&title=Special%3ASearch&fulltext=1&ns0=1&cirrusDumpQuery

  • Query chosen from logstash, with words changed. Logging makes it look like the warning comes from the query generated by second chance search with PhraseSuggestFallbackMethod

MediaSearch reproduction: https://commons.wikimedia.org/wiki/Special:Search?fulltext=1&search=haslicense:other&ns0=1&ns6=1&ns12=1&ns14=1&ns100=1&ns106=1

  • Not specifically MediaSearch, but MediaSearch uses haslicense keyword which is triggering the warning.

Change 792287 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/extensions/CirrusSearch@master] Resolve additional minimum_should_match deprecation warnings

https://gerrit.wikimedia.org/r/792287

Change 792278 merged by jenkins-bot:

[mediawiki/extensions/WikibaseCirrusSearch@master] haslicense: Apply minimum_should_match for elastic 7.x

https://gerrit.wikimedia.org/r/792278

Change 792287 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] Resolve minimum_should_match warnings during random scoring

https://gerrit.wikimedia.org/r/792287

These patches didn't make the tuesday branch cut, intending to cherry-pick them out to clean up the deprecation logs and make it more clear if we have them mostly resolved now.

Change 792649 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/extensions/CirrusSearch@wmf/1.39.0-wmf.12] Resolve minimum_should_match warnings during random scoring

https://gerrit.wikimedia.org/r/792649

Change 792650 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/extensions/WikibaseCirrusSearch@wmf/1.39.0-wmf.12] haslicense: Apply minimum_should_match for elastic 7.x

https://gerrit.wikimedia.org/r/792650

Change 792650 merged by jenkins-bot:

[mediawiki/extensions/WikibaseCirrusSearch@wmf/1.39.0-wmf.12] haslicense: Apply minimum_should_match for elastic 7.x

https://gerrit.wikimedia.org/r/792650

Change 792649 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@wmf/1.39.0-wmf.12] Resolve minimum_should_match warnings during random scoring

https://gerrit.wikimedia.org/r/792649

Mentioned in SAL (#wikimedia-operations) [2022-05-18T07:32:51Z] <dcausse@deploy1002> Synchronized php-1.39.0-wmf.12/extensions/CirrusSearch/includes/Query/FullTextSimpleMatchQueryBuilder.php: Backport: [[gerrit:792649|Resolve minimum_should_match warnings during random scoring (T288765)]] (duration: 00m 56s)

Mentioned in SAL (#wikimedia-operations) [2022-05-18T07:34:51Z] <dcausse@deploy1002> Synchronized php-1.39.0-wmf.12/extensions/WikibaseCirrusSearch/src/Query/HasLicenseFeature.php: Backport: [[gerrit:792650|haslicense: Apply minimum_should_match for elastic 7.x (T288765)]] (duration: 00m 52s)

Change 812324 had a related patch set uploaded (by Matthias Mullie; author: Matthias Mullie):

[mediawiki/extensions/ImageSuggestions@master] Add script to generate notifications for a given list of users & categories

https://gerrit.wikimedia.org/r/812324

Change 812324 had a related patch set uploaded (by Matthias Mullie; author: Matthias Mullie):

[mediawiki/extensions/ImageSuggestions@master] Add script to generate notifications for a given list of users & categories

https://gerrit.wikimedia.org/r/812324

Ignore above comment. I was looking at this task for other reasons and accidentally ended up copying the wrong id. I have since updated that patch's commit with the correct ticket id.