Page MenuHomePhabricator

ElasticSearch 6 migration plan checklist (search cluster)
Open, NormalPublic

Description

WIP plan please fix:

  • Install & test elasticsearch 5.6.14 to deployment-prep
  • T215931 Upgrade codfw to 5.6.14 (rolling restart)
  • switch traffic to codfw (for testing)
  • T215931 Upgrade eqiad to 5.6.14
  • T216052: Upgrade logstash to 5.6.14 (so that we assure compatibility and api-feature-usage can still be populated once the search cluster is running 6.5)
  • In parallel merge the es6 branch for Elastica and CirrusSearch to master and wait for the train to be stable enough when we are sure it can't be rolled back.
    • ES6 compat code should be running on 1.33.0-wmf.19
  • T199791 release plugins for 6.5.4
  • Install & test elasticsearch 6.5.4 to deployment-prep
  • Upgrade codfw to 6.5.4 (rolling restart)
  • Switch search related traffic to codfw
  • Upgrade eqiad to 6.5.4 (rolling restart)
  • Switch search related traffic back to eqiad

Event Timeline

dcausse created this task.Feb 12 2019, 3:39 PM
dcausse triaged this task as Normal priority.
dcausse updated the task description. (Show Details)
dcausse updated the task description. (Show Details)Feb 12 2019, 3:44 PM

Looks good, and all the detail is much appreciated.

Does it make sense to do any manual regression testing on codfw before upgrading eqiad? Like one or two searches with each keyword or special syntax, a few language-specific queries on particular wikis, etc.

EBernhardson updated the task description. (Show Details)Feb 12 2019, 3:59 PM

Hmm—what about Nori (the Korean analyzer) and LTR? I believe we have to disable LTR for Korean, enable Nori, gather more data, then rebuild the LTR model. Sounds like maybe all of that should wait until after the ES upgrade, even though it means re-indexing Korean wikis at a later date.

Also, are there any other LTR-related issues to be concerned about? Is everything in ES6 compatible with the existing models?

debt renamed this task from Elasticsearch 6 migration plan (search cluster) to ElasticSearch 6 migration plan checklist (search cluster).Feb 14 2019, 7:38 PM
debt moved this task from needs triage to elastic / cirrus on the Discovery-Search board.
dcausse updated the task description. (Show Details)Feb 18 2019, 3:41 PM

Once ES6 is running we should just make sure to disable LTR just before we reindex the korean wikipedia. The migration itself should not affect anything related to LTR.

dcausse updated the task description. (Show Details)Feb 18 2019, 3:43 PM
dcausse updated the task description. (Show Details)
dcausse updated the task description. (Show Details)Feb 18 2019, 3:48 PM
dcausse updated the task description. (Show Details)Feb 21 2019, 9:32 PM
dcausse updated the task description. (Show Details)Feb 25 2019, 7:08 PM