For context: [[ https://opensearch.org/blog/unveiling-opensearch-3-0/ | OpenSearch 3 was released in May 2025 ]].
The OpenSearch team [[ https://docs.opensearch.org/latest/install-and-configure/install-opensearch/debian/ | provides Debian packages ]], but we do not currently use them.
If we wished to mirror these upstream packages to our own APT repository, we would:
* follow the guidance here: [[https://wikitech.wikimedia.org/wiki/Reprepro#Adding_a_new_external_repository|Reprepro#Adding_a_new_external_repository]].
* add an entry to the [[https://github.com/wikimedia/operations-puppet/blob/production/modules/aptrepo/files/updates|aptrepo/files/updates]] file.
Rather than use these packages that are published by the upstream project, we have chosen to build our own packages using the procedure defined here: [[https://wikitech.wikimedia.org/wiki/OpenSearch#Building|OpenSearch#Building]].
The packages are then added to our APT repository using the following component names:
* `thirdparty/opensearch1`
* `thirdparty/opensearch2`
The #observability team is the current owner and maintainer of these locally-built packages, which is natural since they adopted OpenSearch for the observability logging stack before it was used anywhere else.
By now, OpenSearch is also used for a number of other use cases in production:
* [[https://wikitech.wikimedia.org/wiki/Search/CirrusSearch|CirrusSearch]] - See: {T370147}
* [[https://wikitech.wikimedia.org/wiki/Data_Platform/Systems/DataHub|DataHub]] uses an OpenSearch cluster running on 3 Ganeti virtual machines.
There is another use production use-case currently in development:
* [[https://wikitech.wikimedia.org/wiki/Service/IPoid|IPoid]] - See: {T357753}
(NOTE)This new opensearch cluster will be hosted on the dse-k8s clusters and provisioned using the [[https://github.com/opensearch-project/opensearch-k8s-operator|opensearch-k8s-operator]].
Block storage for the opensearch cluster pods will be provided by [[https://wikitech.wikimedia.org/wiki/Data_Platform/Systems/Ceph|Ceph]].
Please see: T362105 for more details about this project.
Additionally, as the OpenSearch on K8s project develops, we've seen [[ https://wikimedia.slack.com/archives/C055QGPTC69/p1756490066226489?thread_ts=1756489984.084399&cid=C055QGPTC69 | developer interest in running newer OpenSearch versions ]].
----
Creating this ticket to:
[] Discuss OpenSearch 3 roadmap w/Observability and agree on a plan
[] Discuss OpenSearch package ownership more generally. Potential questions:
-- Should we (#observability , #discovery-search , and #data-platform-sre ) share responsibility for building/publishing packages?
-- Would we benefit from using the upstream opensearch packages for some or all of the use-cases?
-- Is sharing a repo the best approach, or should we each have our own repos?
-- Could the build process be automated, perhaps with [[ https://gitlab.wikimedia.org/repos/search-platform/opensearch-plugins-deb | CI ]] and/or [[ https://gitlab.wikimedia.org/repos/search-platform/sre/ansible-playbooks/wmf_opensearch_plugin_deploy | an ansible playbook? ]]