Page MenuHomePhabricator
Create Task
Maniphest T268861

CirrusSearch uses Elastica's Match class
Closed, ResolvedPublic
Actions

Description

Similar to T267524: Rename css-sanitizer's ‘Match’ class for PHP 8, CirrusSearch uses Elastica\Query\Match which will cause issues with PHP 8.0 support

Looking Upstream https://github.com/ruflin/Elastica/blob/acce6df/src/Query/Match.php has already been renamed to Elastica\Query\MatchQuery

https://github.com/ruflin/Elastica/commit/66dc2ffbadd607c43393f56468888063594f1a0b#diff-0d333847b3ad43ff883b44b177deec9a0bca2e96cd9b0a0905c404e736d33d56

https://github.com/ruflin/Elastica/pull/1799

No release yet, but 7.1.0 maybe coming.

Backport into 6.x requested at https://github.com/ruflin/Elastica/issues/1879

Similar instance in other extensions/code: T268863: Translate uses Elastica's Match class. T268864: WikibaseCirrusSearch uses Elastica's Match class, T268865: WikibaseLexemeCirrusSearch uses Elastica's Match class, T268866: WikibaseMediaInfo uses Elastica's Match class

Details

SubjectRepoBranchLines +/-
composer: Disable linting the Match.php file that breaks PHP 8.0mediawiki/vendormaster+1 -1
Replace Elastica\Query\Match with Elastica\Query\MatchQuerymediawiki/extensions/CirrusSearchREL1_35+43 -43
Replace Elastica\Query\Match with Elastica\Query\MatchQuerymediawiki/extensions/CirrusSearchREL1_36+45 -45
Replace Elastica\Query\Match with Elastica\Query\MatchQuerymediawiki/extensions/CirrusSearchmaster+45 -45
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedReedyT268861 CirrusSearch uses Elastica's Match class
 ResolvedEBernhardsonT271777 Bump rufin/elastica (and related libraries) to versions that support PHP 8.0
 ResolvedGehelT263142 [EPIC] Upgrade Elasticsearch to version 7.10
 ResolvedEBernhardsonT218994 Deprecation warning on elasticsearch 6
ResolvedEBernhardsonT218995 re-enable deprecation warning logger on elasticsearch once issues are solved
 ResolveddcausseT219265 Elasticsearch 6: _retry_on_conflict is deprecated
 ResolveddcausseT219266 Elasticsearch 6 nested_path and nested_filter are deprecated
 ResolveddcausseT219267 Elasticsearch 6: auto_generate_phrase_queries is deprecated and ineffective
 ResolveddcausseT219268 Elasticsearch 6: the classic similarity is deprecated
 ResolveddcausseT219533 Reindex space less languages wikis to use BM25
 OpenNoneT219534 Test MLR models for zhwiki, jawiki and kowiki
 ResolvedEBernhardsonT219269 Elasticsearch 6: silence deprecation warnings to avoid logspam
 DuplicateNoneT215615 Stop using negative scores for deboosting statements
 ResolvedEBernhardsonT288764 Set include_type_name in all get requests
 Resolved• EJosephT288765 Always provide minimum_should_match in bool queries
 ResolvedEBernhardsonT306123 Ensure mjolnir models have positive scores
 DuplicateNoneT273814 Create ElasticSearch 7.10 image in releng/dev-images
 ResolvedGehelT295666 Upgrade Cirrus elasticsearch clusters to 6.8.23
 Resolved• EJosephT294499 Upgrade Cirrus elasticsearch plugins to 6.8.23
 ResolvedbkingT301954 Upgrade deployment-prep to elasticsearch 6.8.23
 ResolvedbkingT301955 Upgrade relforge to elasticsearch 6.8.23
 ResolvedRKemperT278378 Pull Elasticsearch config out of Spicerack
 ResolvedRKemperT301956 Upgrade cloudelastic to elasticsearch 6.8.23
 ResolvedRKemperT301958 Upgrade Search elasticsearch cluster / codfw to elasticsearch 6.8.23
 ResolvedRKemperT301959 Upgrade Search elasticsearch cluster / eqiad to elasticsearch 6.8.23
 ResolvedGehelT304570 elasticsearch: make rolling upgrade proceed to second host after shards are stable
 ResolvedbkingT289135 Upgrade Cirrus Elasticsearch clusters to Debian Bullseye
 ResolvedbkingT299797 Deploy new bullseye elastic cluster nodes on deployment-prep
 ResolvedbkingT301408 Enable search team SRE access to deployment-prep VMs
 ResolvedbkingT307510 Update tls proxy config for Bullseye
 ResolvedbkingT308770 Reimage relforge elastic hosts from Stretch to Bullseye
 ResolvedbkingT309343 Upgrade cloudelastic clusters to Debian Bullseye
 ResolvedbkingT312298 Update NIC firmware on all Elastic PowerEdge R440 elastic hosts
 ResolvedEBernhardsonT295736 Evaluate deployed elasticsearch plugins for 7.10 compatability
 ResolvedEBernhardsonT295734 Bring up two copies of the CirrusSearch browser integration env in cloud
 ResolvedTJonesT300302 Test Elastic 6.8 language analyzers
 Resolved• ZbyszkoT300833 Address "Log4shell" vulnerability in Elasticsearch 7.10.2
 DeclinedNoneT301724 Sunset ApiFeatureUsage
 DeclinedNoneT302638 Sunset ApiFeatureUsage (TDMP)
 ResolvedTJonesT301131 Test Elastic 7.10 language analyzers
 ResolvedEBernhardsonT317200 Reindex all wikis to fix nnbsp regression
 DeclinedNoneT303443 Ensure that all applications depending on the Search Elasticsearch cluster are compatible with Elasticsearch 7.10
 Resolvedbd808T303444 Ensure Toolhub is compatible with Elasticsearch 7.10
 DeclinedNoneT303445 Ensure Phabricator is compatible with Elasticsearch 7.10
 ResolvedEBernhardsonT304290 Create docker elasticsearch images with wmf search plugins
 DeclinedNoneT304569 Remove row/rack awareness in Elasticsearch config
 ResolveddcausseT305428 Upgrade the Translate TTM Elasticsearch implementation to elasticsearch 6.8 and onwards
ResolvedEBernhardsonT306644 re-run wbsearchentities optimization process
ResolvedEBernhardsonT307931 Run CirrusSearch test suite against Elasticsearch 7.10
 ResolvedEBernhardsonT308676 Elasticsearch 7.10.2 rollout plan
 ResolvedbkingT315604 Upgrade relforge cluster to 7.10.2
 ResolvedGehelT316159 Upgrade cloudelastic cluster to 7.10.2
 ResolvedGehelT316240 Upgrade deployment-prep to Elasticsearch 7.10.2
 ResolvedRKemperT316719 Upgrade codfw cluster to Elasticsearch 7.10.2
 ResolvedBUG REPORTEBernhardsonT317381 Reduction in helpfulness and quantity of autocomplete search results
 ResolvedEBernhardsonT317374 Search now caps total results count at 10k because of elasticsearch 7 upgrade
 ResolvedRKemperT317686 Upgrade eqiad cluster to Elasticsearch 7.10.2
 ResolvedEBernhardsonT313434 Migrate apifeatureusage to elasticsearch 7
 ResolvedkostajhT317513 Flow is incompatible with Elastica - Remove ElasticSearch integration / Flow search
 ResolvedReedyT274877 Update ruflin/elastica to 6.1.3

Event Timeline

Reedy created this task.Nov 27 2020, 1:33 AM
Restricted Application added a project: Discovery-Search. · View Herald TranscriptNov 27 2020, 1:33 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Reedy mentioned this in T268863: Translate uses Elastica's Match class.Nov 27 2020, 1:38 AM
Reedy mentioned this in T268864: WikibaseCirrusSearch uses Elastica's Match class.
Reedy mentioned this in T268865: WikibaseLexemeCirrusSearch uses Elastica's Match class.
Reedy mentioned this in T268866: WikibaseMediaInfo uses Elastica's Match class.Nov 27 2020, 1:40 AM
Reedy added a comment.Nov 27 2020, 4:20 AM

Do we want to ask for a backport to 6.x?

I asked at https://github.com/ruflin/Elastica/pull/1799#issuecomment-734534950

This is potentially blocked on T263142: [EPIC] Upgrade Elasticsearch to version 7.10 (which looks to be making progress) and depending on when the upgrades happen, a bump to using at least 7.1.0 of the Elastica library. Not marking as a blocker just yet (it's not one if it gets backported; if it doesn't get backported, it probably is)

Reedy updated the task description. (Show Details)Nov 27 2020, 4:21 AM
Reedy updated the task description. (Show Details)Nov 30 2020, 4:01 PM
EBernhardson moved this task from needs triage to elastic / cirrus on the Discovery-Search board.Nov 30 2020, 4:46 PM
EBernhardson triaged this task as Medium priority.Nov 30 2020, 4:48 PM
Reedy moved this task from Backlog to Wikimedia production on the PHP 8.0 support board.Dec 8 2020, 2:59 AM
Reedy added a subtask: T271777: Bump rufin/elastica (and related libraries) to versions that support PHP 8.0.Jan 12 2021, 8:44 AM
Reedy changed the status of subtask T271777: Bump rufin/elastica (and related libraries) to versions that support PHP 8.0 from Open to Stalled.Feb 7 2021, 12:28 AM
Reedy changed the task status from Open to Stalled.Feb 7 2021, 12:30 AM
Reedy added a subtask: T274877: Update ruflin/elastica to 6.1.3.Feb 16 2021, 11:51 AM
Reedy mentioned this in T275166: CirrusSearch ParseError: "syntax error, unexpected token "match", expecting ":"" (collision with new PHP8 match keyword?).Feb 18 2021, 10:49 PM
Daimona merged a task: T275166: CirrusSearch ParseError: "syntax error, unexpected token "match", expecting ":"" (collision with new PHP8 match keyword?).Feb 27 2021, 3:34 PM
Daimona added a subscriber: Jcwren.
Reedy closed subtask T274877: Update ruflin/elastica to 6.1.3 as Resolved.Mar 6 2021, 6:36 PM
Reedy changed the task status from Stalled to Open.Mar 6 2021, 6:37 PM
gerritbot added a comment.Aug 18 2021, 12:50 PM

Change 713632 had a related patch set uploaded (by Reedy; author: Reedy):

[mediawiki/extensions/CirrusSearch@master] Replace Elastica\\Query\\Match with Elastica\\Query\\MatchQuery

https://gerrit.wikimedia.org/r/713632

gerritbot added a project: Patch-For-Review.Aug 18 2021, 12:50 PM
gerritbot added a comment.Aug 18 2021, 11:32 PM

Change 713632 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] Replace Elastica\\Query\\Match with Elastica\\Query\\MatchQuery

https://gerrit.wikimedia.org/r/713632

Reedy closed this task as Resolved.Aug 18 2021, 11:34 PM
Reedy claimed this task.
ReleaseTaggerBot added a project: MW-1.37-notes (1.37.0-wmf.20; 2021-08-23).Aug 19 2021, 12:00 AM
gerritbot added a comment.Aug 19 2021, 11:01 AM

Change 713848 had a related patch set uploaded (by Reedy; author: Reedy):

[mediawiki/extensions/CirrusSearch@REL1_36] Replace Elastica\\Query\\Match with Elastica\\Query\\MatchQuery

https://gerrit.wikimedia.org/r/713848

gerritbot added a comment.Aug 19 2021, 11:01 AM

Change 713849 had a related patch set uploaded (by Reedy; author: Reedy):

[mediawiki/extensions/CirrusSearch@REL1_35] Replace Elastica\\Query\\Match with Elastica\\Query\\MatchQuery

https://gerrit.wikimedia.org/r/713849

gerritbot added a comment.Aug 20 2021, 6:19 PM

Change 713848 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@REL1_36] Replace Elastica\\Query\\Match with Elastica\\Query\\MatchQuery

https://gerrit.wikimedia.org/r/713848

gerritbot added a comment.Aug 20 2021, 7:07 PM

Change 713849 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@REL1_35] Replace Elastica\\Query\\Match with Elastica\\Query\\MatchQuery

https://gerrit.wikimedia.org/r/713849

Fogen subscribed.EditedSep 16 2021, 2:04 PM

Hello,

After switching to mediawikia 1.37wmf23 with php 8.010 it seems that elastic is now working correctly.

Nevertheless, an error persists when searching with a ":" character.

[99a80ba4319b973cca1c54a7] /index.php?search=%C3%A9v%C3%A9nement+%3A+c&title=Sp%C3%A9cial%3ARecherche&go=Lire&ns0=1 ParseError: syntax error, unexpected token "match"

Backtrace:

from /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/Elastica/vendor/ruflin/elastica/lib/Elastica/Query/MatchQuery.php(10)

#0 /var/www/vhosts/fallout-wiki.com/httpdocs/vendor/composer/ClassLoader.php(322): Composer\Autoload\includeFile()

#1 /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/CirrusSearch/includes/MetaStore/MetaNamespaceStore.php(95): Composer\Autoload\ClassLoader->loadClass()

#2 /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/CirrusSearch/includes/MetaStore/MetaNamespaceStore.php(79): CirrusSearch\MetaStore\MetaNamespaceStore->queryFilter()

#3 /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/CirrusSearch/includes/Searcher.php(469): CirrusSearch\MetaStore\MetaNamespaceStore->find()

#4 /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/CirrusSearch/includes/Util.php(125): CirrusSearch\Searcher->CirrusSearch\{closure}()

#5 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/poolcounter/PoolCounterWorkViaCallback.php(74): CirrusSearch\Util::CirrusSearch\{closure}()

#6 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/poolcounter/PoolCounterWork.php(162): PoolCounterWorkViaCallback->doWork()

#7 /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/CirrusSearch/includes/Util.php(183): PoolCounterWork->execute()

#8 /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/CirrusSearch/includes/Searcher.php(475): CirrusSearch\Util::doPoolCounterWork()

#9 /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/CirrusSearch/includes/Searcher.php(701): CirrusSearch\Searcher->findNamespace()

#10 /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/CirrusSearch/includes/Hooks.php(531): CirrusSearch\Searcher->updateNamespacesFromQuery()

#11 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/HookContainer/HookContainer.php(338): CirrusSearch\Hooks::onSearchGetNearMatch()

#12 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/HookContainer/HookContainer.php(137): MediaWiki\HookContainer\HookContainer->callLegacyHook()

#13 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/HookContainer/HookRunner.php(3192): MediaWiki\HookContainer\HookContainer->run()

#14 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/search/SearchNearMatcher.php(168): MediaWiki\HookContainer\HookRunner->onSearchGetNearMatch()

#15 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/search/SearchNearMatcher.php(70): SearchNearMatcher->getNearMatchInternal()

#16 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/specials/SpecialSearch.php(341): SearchNearMatcher->getNearMatch()

#17 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/specials/SpecialSearch.php(200): SpecialSearch->goResult()

#18 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/specialpage/SpecialPage.php(647): SpecialSearch->execute()

#19 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/specialpage/SpecialPageFactory.php(1366): SpecialPage->run()

#20 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/MediaWiki.php(314): MediaWiki\SpecialPage\SpecialPageFactory->executePath()

#21 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/MediaWiki.php(925): MediaWiki->performRequest()

#22 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/MediaWiki.php(559): MediaWiki->main()

#23 /var/www/vhosts/fallout-wiki.com/httpdocs/index.php(53): MediaWiki->run()

#24 /var/www/vhosts/fallout-wiki.com/httpdocs/index.php(46): wfIndexMain()

#25 {main}

Reedy added a comment.Sep 16 2021, 3:36 PM

This is technically an upstream issue, but we also don't claim PHP 8.0 support either.

The MatchQuery class was allowed for "earlier" migration of code and not need hacks to have code working needing Match in one version of ruflin/elastica and MatchQuery in others.

If you look at https://github.com/ruflin/Elastica/blob/6.1.5/composer.json#L15 it does explicitly say "php": "^7.0", which means it doesn't support PHP 8.

Reedy mentioned this in T271736: Migrate WMF production from PHP 7.2 to PHP 7.4.Sep 16 2021, 3:39 PM
Fogen added a comment.Sep 22 2021, 12:48 PM
In T268861#7359376, @Reedy wrote:

This is technically an upstream issue, but we also don't claim PHP 8.0 support either.

The MatchQuery class was allowed for "earlier" migration of code and not need hacks to have code working needing Match in one version of ruflin/elastica and MatchQuery in others.

If you look at https://github.com/ruflin/Elastica/blob/6.1.5/composer.json#L15 it does explicitly say "php": "^7.0", which means it doesn't support PHP 8.

Hello,

We use the master branch of eslastica which is well noted as compatible with php 8:

"php": "^7.2 || ^8.0",

https://github.com/ruflin/Elastica/blob/master/composer.json

Reedy added a comment.Sep 22 2021, 1:55 PM

Using other versions of libraries than we have set in composer.json isn't supported by us.

Also, based on your stack trace you don't...

In T268861#7359008, @Fogen wrote:

Hello,

After switching to mediawikia 1.37wmf23 with php 8.010 it seems that elastic is now working correctly.

Nevertheless, an error persists when searching with a ":" character.

[99a80ba4319b973cca1c54a7] /index.php?search=%C3%A9v%C3%A9nement+%3A+c&title=Sp%C3%A9cial%3ARecherche&go=Lire&ns0=1 ParseError: syntax error, unexpected token "match"

Backtrace:

from /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/Elastica/vendor/ruflin/elastica/lib/Elastica/Query/MatchQuery.php(10)

On the 6.1.5 tag, https://github.com/ruflin/Elastica/blob/6.1.5/lib/Elastica/Query/MatchQuery.php#L10 and the line contains match and the path to it matches - lib/Elastica/Query/MatchQuery.php

On master, the path to MatchQuery is completely different and doesn't match your stack trace - https://github.com/ruflin/Elastica/blob/master/src/Query/MatchQuery.php - src/Query/MatchQuery.php

Reedy changed the status of subtask T271777: Bump rufin/elastica (and related libraries) to versions that support PHP 8.0 from Stalled to Open.Oct 8 2021, 2:36 PM
Reedy changed the status of subtask T271777: Bump rufin/elastica (and related libraries) to versions that support PHP 8.0 from Open to Stalled.Dec 17 2021, 4:52 PM
Reedy mentioned this in T248925: Make MediaWiki release tarball compatible with PHP 8.0.Jan 29 2022, 2:10 PM
dcausse changed the status of subtask T271777: Bump rufin/elastica (and related libraries) to versions that support PHP 8.0 from Stalled to Open.May 18 2022, 6:41 PM
Jdforrester-WMF closed subtask T271777: Bump rufin/elastica (and related libraries) to versions that support PHP 8.0 as Resolved.Aug 30 2022, 9:20 PM
gerritbot added a comment.Feb 10 2024, 3:57 PM

Change 1000928 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/vendor@master] composer: Disable linting the Match.php file that breaks PHP 8.0

https://gerrit.wikimedia.org/r/1000928

gerritbot added a comment.Feb 14 2024, 2:25 PM

Change 1000928 merged by jenkins-bot:

[mediawiki/vendor@master] composer: Disable linting the Match.php file that breaks PHP 8.0

https://gerrit.wikimedia.org/r/1000928

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits