Setup elasticsearch on new codfw servers
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• Mathew.onipe
	Nov 23 2018, 12:54 PM

Project Tags

Referenced Files

None

Subscribers

View All 12 Subscribers

Description

Determine elasticsearch clusters each servers will belong to. This should be the same as the current implementation (that is belong to the same clusters the old elastic2001-elastic2018 belonged to)
Add nodes and their corresponding role to puppet(site.pp)
Add nodes to conftool-data/nodes/codfw.yaml
Make sure rack and row information for nodes is updated in hieradata/regex.yaml
Add elastic nodes to puppet hieradata for codfw's cirrus.yaml. elastic hosts should be added to their correct clusters
Run puppet to install elasticsearch instances on the nodes
Confirm that the total number of nodes after adding the new servers is correct.
Confirm that shards are being relocated to the newly added nodes

Details

Related Changes in Gerrit:

Subject	Author	Repo	Branch	Lines +/-
elasticsearch: add new elastic2045-elastic2054	Mathew.onipe	operations/puppet	production	+51 -5
elasticsearch: add new elastic2037-elastic2044	Mathew.onipe	operations/puppet	production	+49 -13
cirrus.yaml: add new elastic2037-elastic2054 to existing clusters	Mathew.onipe	operations/puppet	production	+39 -3

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
		Unknown Object (Task)
		Unknown Object (Task)
Resolved	• Mathew.onipe	T210265 Setup elasticsearch on new codfw servers
Resolved	Gehel	T210450 rack/setup/install elastic203[7-9], elastic204[0-9], elastic205[0-4]

Event Timeline

• Mathew.onipe created this task.Nov 23 2018, 12:54 PM

• Mathew.onipe triaged this task as Medium priority.

• Mathew.onipe created this object in space Restricted Space.

• Mathew.onipe edited parent tasks, added: Unknown Object (Task); removed: Unknown Object (Task).Nov 23 2018, 1:00 PM

• Mathew.onipe updated the task description. (Show Details)Nov 23 2018, 1:05 PM

• Mathew.onipe renamed this task from Determine elasticsearch clusters new servers are going to belong to and update corresponding puppet modules to Setup elasticsearch on new codfw servers.Nov 23 2018, 5:06 PM

• Mathew.onipe updated the task description. (Show Details)

• Mathew.onipe updated the task description. (Show Details)Nov 27 2018, 2:59 AM

• Mathew.onipe updated the task description. (Show Details)Nov 27 2018, 3:02 AM

• Mathew.onipe moved this task from Incoming to not in use - please delete on the Discovery-Search (Current work) board.Nov 28 2018, 10:00 AM

dcausse assigned this task to • Mathew.onipe.Nov 28 2018, 10:07 AM

Gehel added a subtask: T210450: rack/setup/install elastic203[7-9], elastic204[0-9], elastic205[0-4].Nov 29 2018, 9:01 AM

I went ahead and moved this out of S4 (as its not procurement), back into S1, and removed the procurement project.

Mentioned in SAL (#wikimedia-operations) [2018-12-03T15:23:24Z] <gehel> start configuration of elastic2037-2044 (new servers) - T210265

Change 475942 merged by Gehel:
[operations/puppet@production] elasticsearch: add new elastic2037-elastic2044

https://gerrit.wikimedia.org/r/475942

Mentioned in SAL (#wikimedia-operations) [2018-12-03T18:08:15Z] <gehel> add elastic2037 to cirrus eqiad (new server) - T210265

Mentioned in SAL (#wikimedia-operations) [2018-12-04T09:11:12Z] <gehel> add elastic2038 to cirrus eqiad (new server) - T210265

Mentioned in SAL (#wikimedia-operations) [2018-12-04T09:31:32Z] <gehel> add elastic2039-2044 to cirrus eqiad (new server) - T210265

Change 477523 had a related patch set uploaded (by Mathew.onipe; owner: Mathew.onipe):
[operations/puppet@production] elasticsearch: add new elastic2045-elastic2054

https://gerrit.wikimedia.org/r/477523

gerritbot added a project: Patch-For-Review.Dec 4 2018, 12:23 PM

• Mathew.onipe updated the task description. (Show Details)Dec 4 2018, 2:39 PM

Mentioned in SAL (#wikimedia-operations) [2018-12-05T09:32:03Z] <gehel> setting up new elasticsearch servers on codfw - elastic2045-2054 - T210265