Page MenuHomePhabricator
Create Task
Maniphest T215916

ElasticSearch 6 migration plan checklist (search cluster)
Closed, ResolvedPublic
Actions

Description

WIP plan please fix:

  • Install & test elasticsearch 5.6.14 to deployment-prep
  • T215931 Upgrade codfw to 5.6.14 (rolling restart)
  • switch traffic to codfw (for testing)
  • T215931 Upgrade eqiad to 5.6.14
  • T216052: Upgrade logstash to 5.6.14 (so that we assure compatibility and api-feature-usage can still be populated once the search cluster is running 6.5)
  • In parallel merge the es6 branch for Elastica and CirrusSearch to master and wait for the train to be stable enough when we are sure it can't be rolled back.
    • ES6 compat code should be running on 1.33.0-wmf.19
  • T199791 release plugins for 6.5.4
  • Install & test elasticsearch 6.5.4 to deployment-prep
  • Upgrade codfw to 6.5.4 (rolling restart)
  • Switch search related traffic to codfw
  • Upgrade eqiad to 6.5.4 (rolling restart)
  • Switch search related traffic back to eqiad

Related Objects
Search...

Event Timeline

dcausse triaged this task as Medium priority.Feb 12 2019, 3:39 PM
dcausse created this task.
dcausse updated the task description. (Show Details)
dcausse updated the task description. (Show Details)Feb 12 2019, 3:44 PM
TJones added a comment.Feb 12 2019, 3:50 PM

Looks good, and all the detail is much appreciated.

Does it make sense to do any manual regression testing on codfw before upgrading eqiad? Like one or two searches with each keyword or special syntax, a few language-specific queries on particular wikis, etc.

EBernhardson updated the task description. (Show Details)Feb 12 2019, 3:59 PM
TJones added a comment.Feb 12 2019, 4:49 PM

Hmm—what about Nori (the Korean analyzer) and LTR? I believe we have to disable LTR for Korean, enable Nori, gather more data, then rebuild the LTR model. Sounds like maybe all of that should wait until after the ES upgrade, even though it means re-indexing Korean wikis at a later date.

Also, are there any other LTR-related issues to be concerned about? Is everything in ES6 compatible with the existing models?

debt renamed this task from Elasticsearch 6 migration plan (search cluster) to ElasticSearch 6 migration plan checklist (search cluster).Feb 14 2019, 7:38 PM
debt moved this task from needs triage to elastic / cirrus on the Discovery-Search board.
dcausse updated the task description. (Show Details)Feb 18 2019, 3:41 PM

Once ES6 is running we should just make sure to disable LTR just before we reindex the korean wikipedia. The migration itself should not affect anything related to LTR.

dcausse updated the task description. (Show Details)Feb 18 2019, 3:43 PM
dcausse updated the task description. (Show Details)
dcausse updated the task description. (Show Details)Feb 18 2019, 3:46 PM
dcausse added a subtask: T215931: Upgrade elasticsearch to 5.6.14.
dcausse updated the task description. (Show Details)Feb 18 2019, 3:48 PM
dcausse updated the task description. (Show Details)Feb 21 2019, 9:32 PM
EBernhardson updated the task description. (Show Details)Feb 22 2019, 12:57 AM
dcausse updated the task description. (Show Details)Feb 25 2019, 7:08 PM
debt closed subtask T215931: Upgrade elasticsearch to 5.6.14 as Resolved.Mar 8 2019, 6:02 PM
Gehel closed this task as Resolved.Jul 11 2019, 5:09 PM
Gehel claimed this task.

ES6 migration has been completed for some time

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL