toolhub: unclear about the procedure here (perhaps quick enough to reindex that it's fine to have a small downtime?)

Downtime during a rebuild is what has been discussed in the past for Toolhub index rebuilds. The dataset is relatively tiny. There was some day dreaming years ago about running an {elastic|open}search node as a Kubernetes pod with a persistent volume claim as the backend as a way to separate from the Cirrus clusters. I never heard if things advanced to a point where that would be possible. It would likely need the project to move out of the wikikube cluster as part of that change too. Toolhub borrowed infrastructure from all over to get into prod.

In T423993#11845280, @bd808 wrote:

toolhub: unclear about the procedure here (perhaps quick enough to reindex that it's fine to have a small downtime?)

Downtime during a rebuild is what has been discussed in the past for Toolhub index rebuilds. The dataset is relatively tiny. There was some day dreaming years ago about running an {elastic|open}search node as a Kubernetes pod with a persistent volume claim as the backend as a way to separate from the Cirrus clusters. I never heard if things advanced to a point where that would be possible. It would likely need the project to move out of the wikikube cluster as part of that change too. Toolhub borrowed infrastructure from all over to get into prod.

It's quite recent but the dse-k8s cluster can now host opensearch clusters, ipoid and the recent semanticsearch experiment have been using this, some doc at https://wikitech.wikimedia.org/wiki/Data_Platform/Systems/OpenSearch-on-K8s.

Mentioned in SAL (#wikimedia-cloud) [2026-04-21T16:31:33Z] <dcausse> closing the toolhub_tools index in the cirrussearch opensearch@codfw (T423993)

@bd808 you may be recalling this Google doc which we started about 2 years ago.

It took quite awhile, but we did finally get the platform launched earlier this year. I'm happy to get together with you and sketch out a migration plan for toolhub at your convenience. Feel free to to read through the doc David linked above and ping me here or in Slack/IRC if you have questions. I'll create a ticket for the migration later this week and CC you if that works.

Mentioned in SAL (#wikimedia-cloud) [2026-04-24T08:36:55Z] <dcausse> remove the toolhub_tools index from the cirrussearch opensearch cluster in codfw (T423993)