Page MenuHomePhabricator

Setup elasticsearch on new codfw servers
Closed, ResolvedPublic


  • Determine elasticsearch clusters each servers will belong to. This should be the same as the current implementation (that is belong to the same clusters the old elastic2001-elastic2018 belonged to)
  • Add nodes and their corresponding role to puppet(site.pp)
  • Add nodes to conftool-data/nodes/codfw.yaml
  • Make sure rack and row information for nodes is updated in hieradata/regex.yaml
  • Add elastic nodes to puppet hieradata for codfw's cirrus.yaml. elastic hosts should be added to their correct clusters
  • Run puppet to install elasticsearch instances on the nodes
  • Confirm that the total number of nodes after adding the new servers is correct.
  • Confirm that shards are being relocated to the newly added nodes

Related Objects

Resolved Mathew.onipe

Event Timeline

Mathew.onipe created this task.
Mathew.onipe created this object in space Restricted Space.
Mathew.onipe edited parent tasks, added: Unknown Object (Task); removed: Unknown Object (Task).Nov 23 2018, 1:00 PM
Mathew.onipe renamed this task from Determine elasticsearch clusters new servers are going to belong to and update corresponding puppet modules to Setup elasticsearch on new codfw servers.Nov 23 2018, 5:06 PM
Mathew.onipe updated the task description. (Show Details)
RobH shifted this object from the Restricted Space space to the S1 Public space.Nov 29 2018, 4:43 PM
RobH removed a project: procurement.

I went ahead and moved this out of S4 (as its not procurement), back into S1, and removed the procurement project.

Mentioned in SAL (#wikimedia-operations) [2018-12-03T15:23:24Z] <gehel> start configuration of elastic2037-2044 (new servers) - T210265

Change 475942 merged by Gehel:
[operations/puppet@production] elasticsearch: add new elastic2037-elastic2044

Mentioned in SAL (#wikimedia-operations) [2018-12-03T18:08:15Z] <gehel> add elastic2037 to cirrus eqiad (new server) - T210265

Mentioned in SAL (#wikimedia-operations) [2018-12-04T09:11:12Z] <gehel> add elastic2038 to cirrus eqiad (new server) - T210265

Mentioned in SAL (#wikimedia-operations) [2018-12-04T09:31:32Z] <gehel> add elastic2039-2044 to cirrus eqiad (new server) - T210265

Change 477523 had a related patch set uploaded (by Mathew.onipe; owner: Mathew.onipe):
[operations/puppet@production] elasticsearch: add new elastic2045-elastic2054

Mentioned in SAL (#wikimedia-operations) [2018-12-05T09:32:03Z] <gehel> setting up new elasticsearch servers on codfw - elastic2045-2054 - T210265

Change 477523 merged by Gehel:
[operations/puppet@production] elasticsearch: add new elastic2045-elastic2054