Page MenuHomePhabricator

Adjust cirrus-integration-test-runner to boot opensearch
Closed, ResolvedPublic

Description

This uses a docker image (built from https://gitlab.wikimedia.org/repos/search-platform/cirrussearch-elasticsearch-image) to run elasticsearch. Put together a comparable image for opensearch and get it working with the integration testing environment. Initially this can leave custom plugins disabled, Cirrus is designed to work without them.

Probably needs a new branch created in CirrusSearch to hold necessary changes. Probably requires forking Elastica somewhere (perhaps into the Elastica extension repo?) and making other necessary changes to allow communication.

Event Timeline

Gehel triaged this task as Medium priority.Aug 28 2024, 8:36 AM
Gehel edited projects, added Discovery-Search (Current work); removed Discovery-Search.

Change #1079333 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[operations/software/elasticsearch/plugins@master] Migrate package to opensearch

https://gerrit.wikimedia.org/r/1079333

Change #1079333 abandoned by Ebernhardson:

[operations/software/elasticsearch/plugins@master] Migrate package to opensearch

Reason:

meant to create in a new repo for opensearch

https://gerrit.wikimedia.org/r/1079333

Change #1079355 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/extensions/CirrusSearch@opensearch] Bare bones OpenSearch 1.3.19 support

https://gerrit.wikimedia.org/r/1079355

Pieces so far:

  • Replace operations/software/elasticsearch/plugins repo with operations/software/opensearch/plugins. Subitted request to create new repo. Plan is to import the old repo to the new name, then archive the old name once we are fully migrated off. Have a patch prepared that will build the .deb with opensearch plugins.
  • MR above to repos/search-platform/cirrussearch-elasticsearch-image adjusts the build to work from the opensearch 1.3.19 base image. Expects to download the opensearch deb from the updated name, or can manually provide the deb created above.
  • New branch, opensearch, in mediawiki/extensions/CirrusSearch. Future patches to opensearch w/ cirrus are intended to go here. This mimics how we have handled migrations in the past with a branch for development.
  • At the moment the current version of the our query dsl, Elastica, appears to be working. Expecting that to change in the future and require forking the client library.

Change #1079355 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@opensearch] Bare bones OpenSearch 1.3.19 support

https://gerrit.wikimedia.org/r/1079355

Change #1080749 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[operations/software/opensearch/plugins@master] Migrate package to opensearch

https://gerrit.wikimedia.org/r/1080749

Change #1081199 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/extensions/CirrusSearch@opensearch] Exclude connections from cirrus-config-dump

https://gerrit.wikimedia.org/r/1081199

Change #1081242 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/extensions/CirrusSearch@master] Bare bones OpenSearch 1.3.19 support

https://gerrit.wikimedia.org/r/1081242

Change #1081243 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/extensions/CirrusSearch@master] Exclude connections from cirrus-config-dump

https://gerrit.wikimedia.org/r/1081243

Change #1081243 abandoned by Ebernhardson:

[mediawiki/extensions/CirrusSearch@master] Exclude connections from cirrus-config-dump

Reason:

wrong branch

https://gerrit.wikimedia.org/r/1081243

Change #1081242 abandoned by Ebernhardson:

[mediawiki/extensions/CirrusSearch@master] Bare bones OpenSearch 1.3.19 support

Reason:

wrong branch

https://gerrit.wikimedia.org/r/1081242

Change #1080355 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[search/extra@master] Update to OpenSearch 1.3.19

https://gerrit.wikimedia.org/r/1080355

Change #1080352 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[search/extra-analysis@master] Update to OpenSearch 1.3.19

https://gerrit.wikimedia.org/r/1080352

Change #1080384 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[search/highlighter@master] [WIP] Update to OpenSearch 1.3.19

https://gerrit.wikimedia.org/r/1080384

Change #1081199 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@opensearch] Exclude connections from cirrus-config-dump

https://gerrit.wikimedia.org/r/1081199

Change #1080355 merged by jenkins-bot:

[search/extra@master] Update to OpenSearch 1.3.19

https://gerrit.wikimedia.org/r/1080355

Change #1085484 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/extensions/CirrusSearch@opensearch] [WIP] Rename experimental-highlighter plugin

https://gerrit.wikimedia.org/r/1085484

Change #1080352 merged by jenkins-bot:

[search/extra-analysis@master] Update to OpenSearch 1.3.19

https://gerrit.wikimedia.org/r/1080352

Change #1080384 merged by jenkins-bot:

[search/highlighter@master] Update to OpenSearch 1.3.19

https://gerrit.wikimedia.org/r/1080384

Change #1092938 had a related patch set uploaded (by Bking; author: Bking):

[operations/puppet@production] opensearch: add components for bullseye and bookworm

https://gerrit.wikimedia.org/r/1092938

Change #1092938 merged by Bking:

[operations/puppet@production] opensearch: add components for bullseye and bookworm

https://gerrit.wikimedia.org/r/1092938

Change #1080749 merged by Ebernhardson:

[operations/software/opensearch/plugins@master] Migrate package to opensearch

https://gerrit.wikimedia.org/r/1080749

Change #1093978 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/extensions/CirrusSearch@master] Bare bones OpenSearch 1.3.x support

https://gerrit.wikimedia.org/r/1093978

Change #1093979 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/extensions/CirrusSearch@master] Exclude connections from cirrus-config-dump

https://gerrit.wikimedia.org/r/1093979

Cindy has made the first V+1 vote, in https://gerrit.wikimedia.org/r/1093978, while running the new opensearch container.

Change #1093978 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] Bare bones OpenSearch 1.3.x support

https://gerrit.wikimedia.org/r/1093978

Change #1093979 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] Exclude connections from cirrus-config-dump

https://gerrit.wikimedia.org/r/1093979