Page MenuHomePhabricator

Migrate elasticsearch scripts to spicerack cookbooks
Closed, ResolvedPublic

Description

The current scripts used for some admin operations on elasticsearch are using cumin directly. Now that we have spicerack as a nice abstraction on top of those, we should use it. The current scripts are available at https://github.com/gehel/estools/ (yes I know, those should not be on github, but in the WMF repos).

Steps:

  • migrate supporting functions into a new elasticsearch module inside spicerack
  • reimplement the top level scripts as spicerack cookbooks

Event Timeline

Change 456322 had a related patch set uploaded (by Mathew.onipe; owner: Mathew.onipe):
[operations/software/spicerack@master] Elasticsearch module is coming up.

https://gerrit.wikimedia.org/r/456322

Change 461267 had a related patch set uploaded (by Mathew.onipe; owner: Mathew.onipe):
[operations/software/spicerack@master] Added force shard allocation to elasticsearch_cluster

https://gerrit.wikimedia.org/r/461267

Change 462514 had a related patch set uploaded (by Mathew.onipe; owner: Mathew.onipe):
[operations/cookbooks@master] Add elasticsearch_operations

https://gerrit.wikimedia.org/r/462514

Change 456322 merged by jenkins-bot:
[operations/software/spicerack@master] Add elasticsearch_cluster module

https://gerrit.wikimedia.org/r/456322

Change 467964 had a related patch set uploaded (by Mathew.onipe; owner: Mathew.onipe):
[operations/cookbooks@master] elasticsearch: pseudo cookbook for JVM upgrade

https://gerrit.wikimedia.org/r/467964

Change 468558 had a related patch set uploaded (by Mathew.onipe; owner: Mathew.onipe):
[operations/software/spicerack@master] elasticsearch_cluster: multi-cluster support

https://gerrit.wikimedia.org/r/468558

Change 461267 abandoned by Mathew.onipe:
elasticsearch_cluster: adding new features

Reason:
features have been moved to elasticsearch clusters

https://gerrit.wikimedia.org/r/461267

Change 462514 abandoned by Mathew.onipe:
Add elasticsearch

Reason:
Not needed anymore:

https://gerrit.wikimedia.org/r/462514

Change 491255 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/cookbooks@master] elasticsearch: add cookbook for rolling upgrade

https://gerrit.wikimedia.org/r/491255

Change 491254 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/software/spicerack@master] elasticsearch: add methods to upgrade elasticsearch and plugins

https://gerrit.wikimedia.org/r/491254

Change 491255 merged by Gehel:
[operations/cookbooks@master] elasticsearch: add cookbook for rolling upgrade

https://gerrit.wikimedia.org/r/491255

Removing assignee @Mathew.onipe as the user does not seem to be active anymore.

Gehel claimed this task.

Cookbooks are now available for all major operations on elasticsearch. There are still improvements to be made (and there always will be).

Change 491254 abandoned by Gehel:

[operations/software/spicerack@master] elasticsearch: add methods to upgrade elasticsearch and plugins

Reason:

https://gerrit.wikimedia.org/r/491254