As per @dcausse, the required plugins are:
- opensearch-knn (pre-installed in the image)
- opensearch-ml (pre-installed in the image)
- opensearch-neural-search (pre-installed in the image)
- analysis-icu
- opensearch-innerhits (from wikimedia gitlab)
Description
Description
Details
Details
Related Changes in GitLab:
| Title | Reference | Author | Source Branch | Dest Branch | |
|---|---|---|---|---|---|
| Plugins: Add opensearch-innerhits plugin | repos/data-engineering/opensearch!15 | bking | innerhits | main | |
| version 3: Install analysis-icu plugin | repos/data-engineering/opensearch!14 | bking | analysis-icu | main |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | EBernhardson | T413969 Make semantic search accessible through Action API | |||
| Open | None | T414683 Support the deployment of opensearch 3.x clusters | |||
| Resolved | bking | T414693 Build and deploy the opensearch 3.x docker image | |||
| Resolved | bking | T414697 Build the required plugins for opensearch 3 |
Event Timeline
Comment Actions
@dcausse , @EBernhardson and I discussed this on IRC last week. I think the plan is to add the analysis-icu plugin to the existing OpenSearch 3x image.
I've taken this ticket and will start working on this soon.
Comment Actions
It's not ready yet, but i'm also prepping a custom plugin we will build in gitlab. Expecting to have the building and published to gitlab early-ish this week.
Comment Actions
First version of the plugin packaged: https://gitlab.wikimedia.org/repos/search-platform/opensearch-innerhits/-/packages/1959
Comment Actions
Note to self: For the analysis-icu, we will try to follow the convention in our flink image and download from Maven, verifying checksums.
The version of OpenSearch we're currently using in production is 3.3.2, so we'll need to use the exact same plugin version.
Comment Actions
Looks like we don't need to download from maven after all. OpenSearch provides its plugins at https://artifacts.opensearch.org , although this is poorly documented. I've updated the image to include the analysis-icu plugin from there. opensearch-innerhits is next.
Comment Actions
bking merged https://gitlab.wikimedia.org/repos/data-engineering/opensearch/-/merge_requests/14
version 3: Install analysis-icu plugin
Comment Actions
bking merged https://gitlab.wikimedia.org/repos/data-engineering/opensearch/-/merge_requests/15
Plugins: Add opensearch-innerhits plugin
Comment Actions
After the above merge, all plugins should be present in the image. Moving into "needs review" so Search Platform team can verify the plugins are present and working as expected.