Prepare MediaWiki and API for shared Elasticsearch indices
Open, Needs TriagePublic
Actions

Assigned To

Authored By

	Andrew-WMDE
	Apr 2 2024, 8:39 AM

Description

Patches:

Example which uses all patches from above:

wbaas-deploy/pull/1515 | Local example for sharing Elasticsearch indices

Steps:

Create a new wiki if you don't have any existing ones
Run the following only once (not per wiki) to initialize the shared indices:

export WBS_DOMAIN={ANY_EXISTING_WIKI_DOMAIN}
export CLUSTER_NAME=default
export MW_CIRRUSSEARCH_INDEX_BASE_NAME=wiki_1
wbaas-deploy/k8s/jobs/elasticSearchInitJob.sh

Run the following for each existing wiki (and any new wikis created prior to step 2) to create their aliases:

kubectl exec -it deployments/api-scheduler -- bash -c "php artisan job:dispatchNow ElasticSearchAliasInit {WIKI_ID}"

Related Objects

Mentioned In: T363458: ElasticsearchIndexInit jobs run by platform api run twice

Event Timeline

Andrew-WMDE created this task.Apr 2 2024, 8:39 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 2 2024, 8:39 AM

Andrew-WMDE moved this task from To do to Doing on the Wikibase Cloud (Kanban board Q1 2024) board.Apr 2 2024, 8:39 AM

Andrew-WMDE updated the task description. (Show Details)Apr 2 2024, 9:57 AM

Andrew-WMDE moved this task from Doing to In Review on the Wikibase Cloud (Kanban board Q1 2024) board.

Added some review but I'm sorry to say I had some questions which I guess need answering and looking at by someone (@Fring ?) next week while I'm out. I also made a short PR for some diagrams that I think describe the current simplified way we create ES indices. I wondered if adding to these for the new architecture (including creating both the shared indices and the aliases) would help communicate what this will look like: https://github.com/wbstack/api/pull/779

Andrew-WMDE moved this task from In Review to Doing on the Wikibase Cloud (Kanban board Q1 2024) board.Mon, Apr 8, 10:35 AM

Andrew-WMDE moved this task from Doing to In Review on the Wikibase Cloud (Kanban board Q1 2024) board.Tue, Apr 9, 12:36 PM

There is review on all of the mentionend PRs, let me know if there are questions.

conny-kawohl_WMDE edited projects, added Wikibase Cloud (Kanban Board Q2 2024); removed Wikibase Cloud (Kanban board Q1 2024).Wed, Apr 17, 1:50 PM

conny-kawohl_WMDE moved this task from To do to In Review on the Wikibase Cloud (Kanban Board Q2 2024) board.Wed, Apr 17, 1:50 PM

Andrew-WMDE moved this task from In Review to Doing on the Wikibase Cloud (Kanban Board Q2 2024) board.Mon, Apr 22, 9:00 AM

Andrew-WMDE updated the task description. (Show Details)Tue, Apr 23, 10:15 AM

Andrew-WMDE updated the task description. (Show Details)Tue, Apr 23, 11:22 AM

Andrew-WMDE moved this task from Doing to In Review on the Wikibase Cloud (Kanban Board Q2 2024) board.Tue, Apr 23, 12:15 PM

Tarrow mentioned this in T363458: ElasticsearchIndexInit jobs run by platform api run twice.Thu, Apr 25, 10:33 AM

Andrew-WMDE updated the task description. (Show Details)Thu, Apr 25, 11:47 AM

Andrew-WMDE updated the task description. (Show Details)Thu, Apr 25, 12:51 PM