Setup elasticsearch on new codfw servers
Closed, ResolvedPublic

Description

  • Determine elasticsearch clusters each servers will belong to. This should be the same as the current implementation (that is belong to the same clusters the old elastic2001-elastic2018 belonged to)
  • Add nodes and their corresponding role to puppet(site.pp)
  • Add nodes to conftool-data/nodes/codfw.yaml
  • Make sure rack and row information for nodes is updated in hieradata/regex.yaml
  • Add elastic nodes to puppet hieradata for codfw's cirrus.yaml. elastic hosts should be added to their correct clusters
  • Run puppet to install elasticsearch instances on the nodes
  • Confirm that the total number of nodes after adding the new servers is correct.
  • Confirm that shards are being relocated to the newly added nodes

Related Objects

StatusAssignedTask
ResolvedMathew.onipe
OpenGehel
Mathew.onipe created this object in space Restricted Space.
Mathew.onipe triaged this task as Normal priority.
Mathew.onipe edited parent tasks, added: Unknown Object (Task); removed: Unknown Object (Task).Fri, Nov 23, 1:00 PM
Mathew.onipe updated the task description. (Show Details)Fri, Nov 23, 1:05 PM
Mathew.onipe updated the task description. (Show Details)Fri, Nov 23, 5:06 PM
Mathew.onipe renamed this task from Determine elasticsearch clusters new servers are going to belong to and update corresponding puppet modules to Setup elasticsearch on new codfw servers.
Mathew.onipe updated the task description. (Show Details)Tue, Nov 27, 2:59 AM
Mathew.onipe updated the task description. (Show Details)Tue, Nov 27, 3:02 AM
RobH shifted this object from the Restricted Space space to the S1 Public space.Thu, Nov 29, 4:43 PM
RobH removed a project: procurement.

I went ahead and moved this out of S4 (as its not procurement), back into S1, and removed the procurement project.

Mentioned in SAL (#wikimedia-operations) [2018-12-03T15:23:24Z] <gehel> start configuration of elastic2037-2044 (new servers) - T210265

Change 475942 merged by Gehel:
[operations/puppet@production] elasticsearch: add new elastic2037-elastic2044

https://gerrit.wikimedia.org/r/475942

Mentioned in SAL (#wikimedia-operations) [2018-12-03T18:08:15Z] <gehel> add elastic2037 to cirrus eqiad (new server) - T210265

Mentioned in SAL (#wikimedia-operations) [2018-12-04T09:11:12Z] <gehel> add elastic2038 to cirrus eqiad (new server) - T210265

Mentioned in SAL (#wikimedia-operations) [2018-12-04T09:31:32Z] <gehel> add elastic2039-2044 to cirrus eqiad (new server) - T210265

Change 477523 had a related patch set uploaded (by Mathew.onipe; owner: Mathew.onipe):
[operations/puppet@production] elasticsearch: add new elastic2045-elastic2054

https://gerrit.wikimedia.org/r/477523

Mathew.onipe updated the task description. (Show Details)Tue, Dec 4, 2:39 PM

Mentioned in SAL (#wikimedia-operations) [2018-12-05T09:32:03Z] <gehel> setting up new elasticsearch servers on codfw - elastic2045-2054 - T210265

Change 477523 merged by Gehel:
[operations/puppet@production] elasticsearch: add new elastic2045-elastic2054

https://gerrit.wikimedia.org/r/477523

Mathew.onipe updated the task description. (Show Details)Mon, Dec 10, 4:34 PM
Mathew.onipe moved this task from In progress to Done on the Discovery-Search (Current work) board.
debt closed this task as Resolved.Mon, Dec 10, 9:07 PM