Page MenuHomePhabricator

Test spicerack elasticsearch module
Closed, ResolvedPublic

Description

This will serve as an integration test to test this module.

Test spicerack elasticsearch module on cumin host(s).

  • Using python repl test elasticsearch_clusters module. Include result or result summary here.
  • Test rolling restart cookbook using relforge as entry point.

Event Timeline

Mathew.onipe triaged this task as Medium priority.Oct 25 2018, 3:38 AM
Mathew.onipe created this task.
Restricted Application removed a project: Patch-For-Review. · View Herald TranscriptOct 25 2018, 3:38 AM
Mathew.onipe renamed this task from Test spicerack elasticsearch module on relforge or similar environment to Test spicerack elasticsearch module on relforge.Jan 8 2019, 3:08 AM
Mathew.onipe renamed this task from Test spicerack elasticsearch module on relforge to Test spicerack elasticsearch module.Jan 10 2019, 1:52 PM
Mathew.onipe updated the task description. (Show Details)

Gehel did some testing and here is the link to the results: https://etherpad.wikimedia.org/p/spicerack-elastic-test-plan
The summary are as follows:

  • Need to fix .get_next_clusters_nodes(datetime.utcnow(), 0) calling this method with nodes_per_run as 0, throws error. The error raised here should be caught and reraised to alert the user/client.
  • Fix .frozen_writes(reason) method. Currently it throws an error about an unexpected keywork arg - doc_id.
  • Fix .start_elasticsearch(). Here, elasticsearch instances were not started. wildcard may not be working the way we think it is working. more investigation to follow.

More test to follow after this before making patches to fix these.

Restricted Application edited projects, added Discovery-Search; removed Discovery-Search (Current work). · View Herald TranscriptJan 24 2019, 3:38 AM

Change 486858 had a related patch set uploaded (by Mathew.onipe; owner: Mathew.onipe):
[operations/software/spicerack@master] elasticsearch_cluster: fix issues from test result

https://gerrit.wikimedia.org/r/486858

Change 489628 had a related patch set uploaded (by Mathew.onipe; owner: Mathew.onipe):
[operations/cookbooks@master] add temp lvs enabled option

https://gerrit.wikimedia.org/r/489628

Change 489628 merged by Gehel:
[operations/cookbooks@master] add temporary lvs enabled option

https://gerrit.wikimedia.org/r/489628

Change 486858 merged by Gehel:
[operations/software/spicerack@master] elasticsearch_cluster: fix issues from test result

https://gerrit.wikimedia.org/r/486858

Change 490383 had a related patch set uploaded (by Mathew.onipe; owner: Mathew.onipe):
[operations/software/spicerack@master] elasticsearch: add type to delete query

https://gerrit.wikimedia.org/r/490383

Change 490383 merged by Gehel:
[operations/software/spicerack@master] elasticsearch: add doc type to delete query

https://gerrit.wikimedia.org/r/490383

Change 490866 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/software/spicerack@master] elasticsearch: retry on TransportError while waiting for node to be up

https://gerrit.wikimedia.org/r/490866

Change 490866 merged by jenkins-bot:
[operations/software/spicerack@master] elasticsearch: retry on TransportError while waiting for node to be up

https://gerrit.wikimedia.org/r/490866

Upgrade to elasticsearch 5.6.x was performed on relforge with spicerack, so testing is complete. There are always things to improve, but this is working well enough !

Change 491750 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/software/spicerack@master] elasticsearch: access production clusters over HTTPS

https://gerrit.wikimedia.org/r/491750

Change 491750 merged by Gehel:
[operations/software/spicerack@master] elasticsearch: access production clusters over HTTPS

https://gerrit.wikimedia.org/r/491750

Change 491797 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/software/spicerack@master] elasticsearch: support cluster names which have '-' in them

https://gerrit.wikimedia.org/r/491797

Change 491797 merged by Gehel:
[operations/software/spicerack@master] elasticsearch: support cluster names which have '-' in them

https://gerrit.wikimedia.org/r/491797

Change 491803 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/software/spicerack@master] elasticsearch: get_next_clusters_nodes raises ElasticsearchClusterError

https://gerrit.wikimedia.org/r/491803

Change 491808 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/software/spicerack@master] elasticsearch: systemctl iterates explicitly on elasticsearch instances

https://gerrit.wikimedia.org/r/491808

Change 491850 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] elasticsearch: add script to execute systemctl on each elasticsearch instance

https://gerrit.wikimedia.org/r/491850

Change 491803 merged by Gehel:
[operations/software/spicerack@master] elasticsearch: get_next_clusters_nodes raises ElasticsearchClusterError

https://gerrit.wikimedia.org/r/491803

Change 491850 merged by Gehel:
[operations/puppet@production] elasticsearch: add a file with the list on elasticsearch instances

https://gerrit.wikimedia.org/r/491850

Change 491808 merged by Gehel:
[operations/software/spicerack@master] elasticsearch: systemctl iterates explicitly on elasticsearch instances

https://gerrit.wikimedia.org/r/491808

Change 491960 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/software/spicerack@master] elasticsearch: fix typo (xarg instead of xargs)

https://gerrit.wikimedia.org/r/491960

Change 491960 merged by Gehel:
[operations/software/spicerack@master] elasticsearch: fix typo (xarg instead of xargs)

https://gerrit.wikimedia.org/r/491960

debt closed this task as Resolved.Feb 22 2019, 8:39 PM