Maniphest T200198

Drop the "archive" type in the general index
Closed, ResolvedPublic
Actions

Description

The archive type has to be removed prior to upgrading to 6.x.
As we are moving to have multiple elastic instance per node we can create many more indices so the easiest solution is to simply create a new index dedicate to archives.

Details

Subject	Repo	Branch	Lines +/-
Move "archive" to its own index	mediawiki/extensions/CirrusSearch	master	+405 -282
Start using the archive alias	mediawiki/extensions/CirrusSearch	master	+10 -9
Add the "archive" alias	mediawiki/extensions/CirrusSearch	master	+35 -6

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Resolved	EBernhardson	T183281 [epic] ELK upgrade to 6.x (elasticsearch, kibana, logstash)
Resolved	None	T183282 [epic] Search cluster upgrade to 6.x
Resolved	None	T194199 [Epic] Prepare for Elasticsearch 6 upgrade
Resolved	dcausse	T200198 Drop the "archive" type in the general index
Resolved	EBernhardson	T213851 Drop "archive" from generic index on testwiki

Event Timeline

dcausse created this task.Jul 23 2018, 2:20 PM

Restricted Application edited projects, added Discovery-Search; removed Discovery-Search (Current work). · View Herald TranscriptJul 23 2018, 2:20 PM

dcausse claimed this task.Jul 23 2018, 5:27 PM

dcausse edited projects, added Discovery-Search (Current work); removed Discovery-Search.

dcausse moved this task from Incoming to not in use - please delete on the Discovery-Search (Current work) board.

Change 447641 had a related patch set uploaded (by DCausse; owner: DCausse):
[mediawiki/extensions/CirrusSearch@master] Add the "archive" alias

https://gerrit.wikimedia.org/r/447641

Change 447642 had a related patch set uploaded (by DCausse; owner: DCausse):
[mediawiki/extensions/CirrusSearch@master] Move "archive" to its own index

https://gerrit.wikimedia.org/r/447642

Change 448000 had a related patch set uploaded (by DCausse; owner: DCausse):
[mediawiki/extensions/CirrusSearch@master] Start using the archive alias

https://gerrit.wikimedia.org/r/448000

dcausse moved this task from not in use - please delete to Needs review on the Discovery-Search (Current work) board.Jul 26 2018, 4:40 PM

Change 447641 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@master] Add the "archive" alias

https://gerrit.wikimedia.org/r/447641

ReleaseTaggerBot added a project: MW-1.32-notes (WMF-deploy-2018-07-31 (1.32.0-wmf.15)).Jul 26 2018, 10:00 PM

EBernhardson moved this task from Needs review to Waiting on the Discovery-Search (Current work) board.Jul 30 2018, 10:54 PM

Moving to waiting/blocked, I'm thinking we want to delay creating the archive indices until after we deploy multiple elasticsearch instances? We probably could deploy it before hand, the master node latency evaluation was reasonable, not great but reasonable, with only the new archive indices added. With 2x the archive indices added was where we started seeing more erratic master latency behaviour.

Safest though would be to let this be waiting/blocked until then.

EBernhardson mentioned this in T199256: Setup a cirrussearch branch for es 6.x and make it work.Aug 1 2018, 9:07 PM

Change 448000 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@master] Start using the archive alias

https://gerrit.wikimedia.org/r/448000

ReleaseTaggerBot edited projects, added MW-1.32-notes (WMF-deploy-2018-08-07 (1.32.0-wmf.16)); removed MW-1.32-notes (WMF-deploy-2018-07-31 (1.32.0-wmf.15)).Aug 3 2018, 6:01 PM

This is finishing up work started in T199256.

Change 447642 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@master] Move "archive" to its own index

https://gerrit.wikimedia.org/r/447642

ReleaseTaggerBot added a project: MW-1.33-notes (1.33.0-wmf.14; 2019-01-22).Jan 22 2019, 6:00 PM

dcausse moved this task from Waiting to Needs Reporting on the Discovery-Search (Current work) board.Jan 22 2019, 6:52 PM

debt closed this task as Resolved.Jan 26 2019, 7:27 PM

debt closed subtask T213851: Drop "archive" from generic index on testwiki as Resolved.Feb 7 2019, 9:21 PM