Page MenuHomePhabricator
Create Task
Maniphest T273814

Create ElasticSearch 7.10 image in releng/dev-images
Closed, DuplicatePublic
Actions

Description

(Feel free to remove this as a subtask!)

We have a straightforward image for ElasticSearch 6 in the releng/dev-images repo added as part of T231864: Create elasticsearch service, this task would have us create a version for ElasticSearch 7.

I suppose that is probably blocked on the availability of the wikimedia/search/extra and wikimedia/search/highlighter/experimental-highlighter-elasticsearch-plugin plugins for ElasticSearch 7, but maybe ElasticSearch 7 without those plugins would be usable in some form?

Related Objects
Search...

Event Timeline

kostajh created this task.Feb 3 2021, 8:25 PM
kostajh renamed this task from Create ElasticSearch 7.x image to Create ElasticSearch 7.x image in releng/dev-images.Feb 3 2021, 8:28 PM
kostajh added a project: dev-images.
EBernhardson subscribed.Feb 8 2021, 8:26 PM

CirrusSearch works without the search-extra and highlighter plugins, $wgCirrusSearchWikimediaExtraPlugin = [] and $wgCirrusSearchUseExperimentalHighlighter = false respectively. These are the defaults, but dev environments tend to pull in FullyFeaturedConfig.php that turn them on. I don't think this needs to block putting toghether images, but depends on the intended use case.

kostajh added a comment.Feb 12 2021, 6:35 PM
In T273814#6812832, @EBernhardson wrote:

CirrusSearch works without the search-extra and highlighter plugins, $wgCirrusSearchWikimediaExtraPlugin = [] and $wgCirrusSearchUseExperimentalHighlighter = false respectively. These are the defaults, but dev environments tend to pull in FullyFeaturedConfig.php that turn them on. I don't think this needs to block putting toghether images, but depends on the intended use case.

Good to know, thanks!

Is CirrusSearch/Elastica currently supposed to work with ElasticSearch 7, or is there code in progress to support that?

When I try to use the elastic search 7 oss image (docker.elastic.co/elasticsearch/elasticsearch-oss:7.10.2-arm64) I get this error when attempting to create the index:

docker-compose exec mediawiki php extensions/CirrusSearch/maintenance/UpdateSearchIndexConfig.php
Updating cluster ...
indexing namespaces...
mw_cirrus_metastore missing, creating new metastore index.
Creating metastore index... mw_cirrus_metastore_first	Scanning available plugins...none
Elastica\Exception\ResponseException from line 182 of /var/www/html/w/vendor/ruflin/elastica/lib/Elastica/Transport/Http.php: Root mapping definition has unsupported parameters:  [mw_cirrus_metastore : {dynamic=false, properties={mediawiki_version={type=keyword}, mapping_min={type=long}, analysis_maj={type=long}, cirrus_commit={type=keyword}, mapping_maj={type=long}, wiki={type=keyword}, shard_count={type=long}, type={type=keyword}, index_name={type=keyword}, mediawiki_commit={type=keyword}, analysis_min={type=long}, namespace_name={norms=false, analyzer=near_match_asciifolding, type=text, index_options=docs}}}] [reason: Failed to parse mapping [_doc]: Root mapping definition has unsupported parameters:  [mw_cirrus_metastore : {dynamic=false, properties={mediawiki_version={type=keyword}, mapping_min={type=long}, analysis_maj={type=long}, cirrus_commit={type=keyword}, mapping_maj={type=long}, wiki={type=keyword}, shard_count={type=long}, type={type=keyword}, index_name={type=keyword}, mediawiki_commit={type=keyword}, analysis_min={type=long}, namespace_name={norms=false, analyzer=near_match_asciifolding, type=text, index_options=docs}}}]]
#0 /var/www/html/w/vendor/ruflin/elastica/lib/Elastica/Request.php(194): Elastica\Transport\Http->exec(Object(Elastica\Request), Array)
#1 /var/www/html/w/vendor/ruflin/elastica/lib/Elastica/Client.php(689): Elastica\Request->send()
#2 /var/www/html/w/vendor/ruflin/elastica/lib/Elastica/Index.php(571): Elastica\Client->request('mw_cirrus_metas...', 'PUT', Array, Array)
#3 /var/www/html/w/extensions/CirrusSearch/includes/MetaStore/MetaStoreIndex.php(237): Elastica\Index->request('mw_cirrus_metas...', 'PUT', Array, Array)
#4 /var/www/html/w/extensions/CirrusSearch/includes/MetaStore/MetaStoreIndex.php(169): CirrusSearch\MetaStore\MetaStoreIndex->createNewIndex()
#5 /var/www/html/w/extensions/CirrusSearch/includes/Maintenance/Maintenance.php(218): CirrusSearch\MetaStore\MetaStoreIndex->createIfNecessary()
#6 /var/www/html/w/extensions/CirrusSearch/maintenance/IndexNamespaces.php(41): CirrusSearch\Maintenance\Maintenance->maybeCreateMetastore()
#7 /var/www/html/w/extensions/CirrusSearch/maintenance/UpdateSearchIndexConfig.php(66): CirrusSearch\Maintenance\IndexNamespaces->execute()
#8 /var/www/html/w/maintenance/doMaintenance.php(106): CirrusSearch\Maintenance\UpdateSearchIndexConfig->execute()
#9 /var/www/html/w/extensions/CirrusSearch/maintenance/UpdateSearchIndexConfig.php(104): require_once('/var/www/html/w...')
EBernhardson added a comment.Feb 12 2021, 6:46 PM

Unlikely it works directly with elastic 7, when they say breaking changes on major versions elastic means business :) We've found it very difficult to support multiple versions at once, we always transition from one to the next.

kostajh changed the task status from Open to Stalled.Feb 23 2021, 8:54 AM

Let's revisit when the overall ES7 upgrade happens.

MPhamWMF triaged this task as Low priority.Mar 1 2021, 4:09 PM
MPhamWMF moved this task from needs triage to elastic / cirrus on the Discovery-Search board.
GreenReaper mentioned this in T272500: Create arm64 image variants of releng/dev-images used by MediaWiki-Docker.Jul 27 2021, 9:20 PM
MPhamWMF renamed this task from Create ElasticSearch 7.x image in releng/dev-images to Create ElasticSearch 7.11 image in releng/dev-images.Oct 26 2021, 9:59 PM
MPhamWMF renamed this task from Create ElasticSearch 7.11 image in releng/dev-images to Create ElasticSearch 7.10 image in releng/dev-images.Oct 27 2021, 9:14 PM
Gehel closed this task as a duplicate of T304290: Create docker elasticsearch images with wmf search plugins.Mar 21 2022, 3:18 PM

More specific work done on T304290 should solve this task as well.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL