[EPIC] Deploy Elasticsearch to WBstack
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• Samantha_Alipio_WMDE
	Jun 17 2021, 10:25 AM

Description

What it entails
Deploying an elasticsearch server/cluster; deploying elasticsearch-related MW extensions; any other automation and/or services needed to make multiple wikis be able to talk to elasticsearch.

Why needed
Elasticsearch is a requirement for Wikibase 1.36 as case insensitive search is not possible without it after wb_terms. It also improves the advanced search capabilities of Wikibase, such as statement value matching etc.

Impact

Can proceed with 1.36 upgrade of WBstack
Improved search capability

Stages

Deploy an elastic search service to wbstack wbstack (WBaaS (Wikibase as a Service) 2021 Elastic Search Focused Mini Hike)
Deploy inital code and configuration to connect mediawiki & WIkibase to elastic search wbstack (WBaaS (Wikibase as a Service) 2021 Elastic Search Focused Mini Hike)
Implement additional things
- Indexes need to be rebuilt when lexeme is enabled T286780: When WikibaseLexeme is enabled on a wbstack site, rebuild elastic indexes if needed
Create new sites with elastic search T286781: Create new wbstack sites with elastic search enabled and indexes built
Convert existing sites to use elastic search https://github.com/wbstack/deploy/issues/21

Related Objects
Search...

Status	Assigned	Task
Resolved	Addshore	T285084 [INITIATIVE] Prepare all work necessary before a 1.36 Upgrade to WBstack
Resolved	Addshore	T285085 [EPIC] Deploy Elasticsearch to WBstack
Resolved	Addshore	T285540 Add ElasticSearch to development docker-compose
Resolved	Addshore	T285541 Enable ElasticSearch integration
Duplicate	None	T285542 Deploy and Enable ElasticSearch on WBStack
Resolved	• toan	T285937 Make WBStack mediawiki docker-compose dev setup work with multiple sites (to test elastic)
Resolved	• toan	T286037 Prepare an elasticsearch service for deployment on wbstack
Resolved	• bete	T286039 Add elastic search index creation to mediawiki setup code for wbstack
Resolved	• toan	T286780 When WikibaseLexeme is enabled on a wbstack site, rebuild elastic indexes if needed
Resolved	• toan	T286781 Create new wbstack sites with elastic search enabled and indexes built

Event Timeline

• Samantha_Alipio_WMDE created this task.Jun 17 2021, 10:25 AM

Addshore edited projects, added wbstack (WBaaS (Wikibase as a Service) 2021 Elastic Search Focused Mini Hike); removed wbstack.Jun 25 2021, 9:52 AM

Addshore closed subtask T285540: Add ElasticSearch to development docker-compose as Resolved.Jul 12 2021, 1:31 PM

Addshore closed subtask T285937: Make WBStack mediawiki docker-compose dev setup work with multiple sites (to test elastic) as Resolved.Jul 12 2021, 8:22 PM

Addshore closed subtask T286039: Add elastic search index creation to mediawiki setup code for wbstack as Resolved.Jul 16 2021, 11:47 AM

Addshore closed subtask T286037: Prepare an elasticsearch service for deployment on wbstack as Resolved.

Addshore closed subtask T285541: Enable ElasticSearch integration as Resolved.

Addshore edited projects, added wbstack; removed wbstack (WBaaS (Wikibase as a Service) 2021 Elastic Search Focused Mini Hike).Jul 16 2021, 11:48 AM

Addshore merged a task: T285542: Deploy and Enable ElasticSearch on WBStack.

Addshore added a subscriber: • toan.

Addshore updated the task description. (Show Details)Jul 16 2021, 11:54 AM

Addshore updated the task description. (Show Details)Jul 16 2021, 11:56 AM

Addshore updated the task description. (Show Details)Jul 16 2021, 12:04 PM

Addshore updated the task description. (Show Details)Jul 16 2021, 12:07 PM

Addshore updated the task description. (Show Details)

Addshore moved this task from Backlog to Ready to estimate on the wbstack board.

Addshore mentioned this in T285084: [INITIATIVE] Prepare all work necessary before a 1.36 Upgrade to WBstack.Jul 16 2021, 12:11 PM

Addshore moved this task from Ready to estimate to Epics n stuff on the wbstack board.

Addshore mentioned this in T284916: Investigate the required effort of using wbstack as a test system.Jul 16 2021, 12:42 PM

@Lucas_Werkmeister_WMDE mentioned another thing in a call today.

With the tickets that are currently written, we currently deal with:

Wiki creation, and creating the needed indexes
Turning on Lexeme, and creating additional indexes

However there are some other cases that we should think about

Turning off Lexeme, and then turning Lexeme back on again
Turning off Lexeme in one version of MW/WB, having an update happen (that might change indexes), and turning Lexeme back on again

We should consider the 2 further cases and decide the best action, this could for example be making lexeme a one way switch? OR rather than load / unload the extension, instead just enable / disable the entity type?

Loz.ross subscribed.Jul 27 2021, 5:37 PM

Addshore closed subtask T286780: When WikibaseLexeme is enabled on a wbstack site, rebuild elastic indexes if needed as Resolved.Aug 24 2021, 1:04 PM

Addshore closed subtask T286781: Create new wbstack sites with elastic search enabled and indexes built as Resolved.Sep 2 2021, 3:03 PM

This was done!

Restricted Application added a project: User-Addshore. · View Herald TranscriptNov 5 2021, 3:12 PM

[EPIC] Deploy Elasticsearch to WBstackClosed, ResolvedPublicActions

Description

Related ObjectsSearch...

Event Timeline

[EPIC] Deploy Elasticsearch to WBstack
Closed, ResolvedPublic
Actions

Related Objects
Search...