In order to upgrade the completion suggester to have real time index updates, and to improve performance and stability, we want to upgrade the Wikimedia search cluster to use Elasticsearch 2.3. Upgrading to Elasticsearch 2.3 is also, practically speaking, a pre-requisite to upgrading to Elasticsearch 5 in the future. Elasticsearch 5 will allow us to experiment with BM25 and see if it's better than tf-idf. This epic tracks the work required to upgrade to Elasticsearch 2.3.
Here's the chronology of the subtasks:
- T131941: EPIC: upgrade wikimedia elasticsearch plugins to elasticsearch 2.3.1
- T133121: Upgrade CirrusSearch to use Elastica 3.1
- T133123: Create test instance in Labs which uses Elasticsearch 2.3 so we can test it and find what's broken
- T133124: Devise a plan on how to upgrade to Elasticsearch 2.3 without turning user-facing search features off
- T133125: Upgrade codfw data centre to Elasticsearch 2.3
- T133126: Upgrade eqiad data centre to Elasticsearch 2.3
- 3 and 4 can be done in parallel, but both are blocked by 2, and both block 5 and 6.
- 5 and 6 can be done in parallel in theory, but in practice it'd be better if we did them separately.