Page MenuHomePhabricator

Configure elasticsearch crosscluster on production search servers
Closed, ResolvedPublic

Description

For now we will be using persistent configuration in the cluster settings.

I created a small python script on mwmaint1002:~/home/dcausse/es_multi_instance/push_cross_cluster_conf.py and can be used like this:

python push_cross_cluster_conf.py https://search.svc.codfw.wmnet:9443/_cluster/settings --ccc psi=psi_codfw_masters.lst chi=chi_codfw_masters.lst

Event Timeline

dcausse triaged this task as Normal priority.Jan 8 2019, 9:10 AM
dcausse created this task.
dcausse renamed this task from Configure cross cluster on production servers to Configure elasticsearch crosscluster on production search servers.Jan 8 2019, 9:13 AM

Change 482770 had a related patch set uploaded (by DCausse; owner: DCausse):
[mediawiki/extensions/CirrusSearch@master] Add scripts/push_cross_cluster_conf.py

https://gerrit.wikimedia.org/r/482770

Mentioned in SAL (#wikimedia-operations) [2019-01-08T10:34:16Z] <dcausse> elastic@eqiad setting crosscluster conf on production search cluster (T213150)

omega@codfw

curl -s https://search.svc.codfw.wmnet:9443/_cluster/settings | jq .persistent.search.remote

{
  "chi": {
    "seeds": [
      "elastic2042.codfw.wmnet:9300",
      "elastic2025.codfw.wmnet:9300",
      "elastic2031.codfw.wmnet:9300"
    ]
  },
  "psi": {
    "seeds": [
      "elastic2027.codfw.wmnet:9700",
      "elastic2029.codfw.wmnet:9700",
      "elastic2048.codfw.wmnet:9700"
    ]
  }
}

psi@codfw

curl -s https://search.svc.codfw.wmnet:9643/_cluster/settings | jq .persistent.search.remote

{
  "chi": {
    "seeds": [
      "elastic2042.codfw.wmnet:9300",
      "elastic2025.codfw.wmnet:9300",
      "elastic2031.codfw.wmnet:9300"
    ]
  },
  "omega": {
    "seeds": [
      "elastic2042.codfw.wmnet:9500",
      "elastic2047.codfw.wmnet:9500",
      "elastic2038.codfw.wmnet:9500"
    ]
  }
}

chi@codfw

curl -s https://search.svc.codfw.wmnet:9243/_cluster/settings | jq .persistent.search.remote

{
  "omega": {
    "seeds": [
      "elastic2042.codfw.wmnet:9500",
      "elastic2047.codfw.wmnet:9500",
      "elastic2038.codfw.wmnet:9500"
    ]
  },
  "psi": {
    "seeds": [
      "elastic2027.codfw.wmnet:9700",
      "elastic2029.codfw.wmnet:9700",
      "elastic2048.codfw.wmnet:9700"
    ]
  }
}

omega@eqiad

curl -s https://search.svc.eqiad.wmnet:9443/_cluster/settings | jq .persistent.search.remote

{
  "chi": {
    "seeds": [
      "elastic1036.eqiad.wmnet:9300",
      "elastic1030.eqiad.wmnet:9300",
      "elastic1040.eqiad.wmnet:9300"
    ]
  },
  "psi": {
    "seeds": [
      "elastic1052.eqiad.wmnet:9700",
      "elastic1048.eqiad.wmnet:9700",
      "elastic1050.eqiad.wmnet:9700"
    ]
  }
}

psi@eqiad

curl -s https://search.svc.eqiad.wmnet:9643/_cluster/settings | jq .persistent.search.remote

{
  "chi": {
    "seeds": [
      "elastic1036.eqiad.wmnet:9300",
      "elastic1030.eqiad.wmnet:9300",
      "elastic1040.eqiad.wmnet:9300"
    ]
  },
  "omega": {
    "seeds": [
      "elastic1034.eqiad.wmnet:9500",
      "elastic1040.eqiad.wmnet:9500",
      "elastic1038.eqiad.wmnet:9500"
    ]
  }
}

chi@eqiad

curl -s https://search.svc.eqiad.wmnet:9243/_cluster/settings | jq .persistent.search.remote

{
  "omega": {
    "seeds": [
      "elastic1034.eqiad.wmnet:9500",
      "elastic1040.eqiad.wmnet:9500",
      "elastic1038.eqiad.wmnet:9500"
    ]
  },
  "psi": {
    "seeds": [
      "elastic1052.eqiad.wmnet:9700",
      "elastic1048.eqiad.wmnet:9700",
      "elastic1050.eqiad.wmnet:9700"
    ]
  }
}

Change 482770 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@master] Add scripts/push_cross_cluster_conf.py

https://gerrit.wikimedia.org/r/482770

debt closed this task as Resolved.Jan 18 2019, 7:15 PM