Page MenuHomePhabricator

EPIC: Upgrade Wikimedia search cluster to use Elasticsearch 2.3
Closed, ResolvedPublic

Description

In order to upgrade the completion suggester to have real time index updates, and to improve performance and stability, we want to upgrade the Wikimedia search cluster to use Elasticsearch 2.3. Upgrading to Elasticsearch 2.3 is also, practically speaking, a pre-requisite to upgrading to Elasticsearch 5 in the future. Elasticsearch 5 will allow us to experiment with BM25 and see if it's better than tf-idf. This epic tracks the work required to upgrade to Elasticsearch 2.3.

Here's the chronology of the subtasks:

  1. T131941: EPIC: upgrade wikimedia elasticsearch plugins to elasticsearch 2.3.1
  2. T133121: Upgrade CirrusSearch to use Elastica 3.1
  3. T133123: Create test instance in Labs which uses Elasticsearch 2.3 so we can test it and find what's broken
  4. T133124: Devise a plan on how to upgrade to Elasticsearch 2.3 without turning user-facing search features off
  5. T133125: Upgrade codfw data centre to Elasticsearch 2.3
  6. T133126: Upgrade eqiad data centre to Elasticsearch 2.3

Notes:

  • 3 and 4 can be done in parallel, but both are blocked by 2, and both block 5 and 6.
  • 5 and 6 can be done in parallel in theory, but in practice it'd be better if we did them separately.

Related Objects

StatusSubtypeAssignedTask
DeclinedNone
Resolveddebt
Resolveddebt
ResolvedGehel
ResolvedRobH
ResolvedEBernhardson
ResolvedEBernhardson
ResolvedEBernhardson
Resolveddebt
Resolveddcausse
Resolveddcausse
Resolveddcausse
Resolveddcausse
Resolveddcausse
Resolveddcausse
Resolveddcausse
Resolveddcausse
Resolveddcausse
ResolvedMaxSem
DeclinedNone

Event Timeline

Deskana updated the task description. (Show Details)
Deskana triaged this task as Medium priority.Apr 19 2016, 11:56 PM
debt claimed this task.