Page MenuHomePhabricator

Productionize es20[26-34] and es10[26-34]
Closed, ResolvedPublic

Description

The following hosts will be used to replace existing es1, es2 and es3 hosts.

eqiad:

  • es1026 (A1) -> es1011 (es2)
  • es1027 (A1) -> es1012 (es1)
  • es1028 (A3) -> es1014 (es3)
  • es1029 (B1) -> es1016 (es1)
  • es1030 (B1) -> es1013 (es2)
  • es1031 (C3) -> es1017 (es3)
  • es1032 (C3) -> es1018 (es1)
  • es1033 (D8) -> es1015 (es2)
  • es1034 (D8) -> es1019 (es3)

codfw:
(reminder: extend their lvm: lvextend -L+1600G /dev/mapper/tank-data && xfs_growfs /srv
(set them to active on netbox)

  • es2026 (A1) -> es2014 (es2)
  • es2027 (A6) -> es2017 (es3)
  • es2028 (B1) -> es2011 (es1)
  • es2029 (B1) -> es2018 (es3)
  • es2030 (C1) -> es2012 (es1)
  • es2031 (C1) -> es2015 (es2)
  • es2032 (D1) -> es2013 (es1)
  • es2033 (D1) -> es2016 (es2)
  • es2034 (A6) -> es2019 (es3)

Details

SubjectRepoBranchLines +/-
operations/puppetproduction+1 -1
operations/puppetproduction+0 -1
operations/puppetproduction+1 -0
operations/puppetproduction+2 -0
operations/puppetproduction+0 -2
operations/puppetproduction+23 -7
operations/puppetproduction+1 -1
operations/puppetproduction+0 -3
operations/puppetproduction+3 -0
operations/puppetproduction+22 -1
operations/puppetproduction+0 -3
operations/puppetproduction+3 -0
operations/puppetproduction+8 -1
operations/puppetproduction+8 -1
operations/puppetproduction+8 -1
operations/puppetproduction+9 -0
operations/dnsmaster+0 -2
operations/puppetproduction+3 -0
operations/puppetproduction+0 -3
operations/puppetproduction+22 -4
operations/puppetproduction+0 -2
operations/puppetproduction+2 -0
operations/puppetproduction+1 -1
operations/puppetproduction+15 -1
operations/puppetproduction+9 -2
operations/puppetproduction+0 -2
operations/puppetproduction+2 -0
operations/puppetproduction+9 -2
operations/puppetproduction+9 -2
operations/puppetproduction+1 -0
operations/puppetproduction+0 -1
operations/puppetproduction+1 -1
operations/puppetproduction+1 -0
operations/puppetproduction+8 -1
Show related patches Customize query in gerrit

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 641328 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] instances.yaml: Add es1033,es1034

https://gerrit.wikimedia.org/r/641328

Change 641328 merged by Marostegui:
[operations/puppet@production] instances.yaml: Add es1033,es1034

https://gerrit.wikimedia.org/r/641328

Mentioned in SAL (#wikimedia-operations) [2020-11-17T06:38:05Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Pool es1033 with minimum weight on es2 T261717', diff saved to https://phabricator.wikimedia.org/P13271 and previous config saved to /var/cache/conftool/dbconfig/20201117-063805-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T06:39:34Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Pool es1034 with minimum weight on es3 T261717', diff saved to https://phabricator.wikimedia.org/P13272 and previous config saved to /var/cache/conftool/dbconfig/20201117-063933-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T06:45:22Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1015 (re)pooling @ 25%: Slowly pool es1015 after cloning es1033 T261717', diff saved to https://phabricator.wikimedia.org/P13273 and previous config saved to /var/cache/conftool/dbconfig/20201117-064522-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T06:45:47Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1019 (re)pooling @ 25%: Slowly pool es1019 after cloning es1034 T261717', diff saved to https://phabricator.wikimedia.org/P13274 and previous config saved to /var/cache/conftool/dbconfig/20201117-064546-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T06:47:06Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1033 (re)pooling @ 10%: Slowly pool es1033 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13275 and previous config saved to /var/cache/conftool/dbconfig/20201117-064705-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T06:47:17Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1034 (re)pooling @ 10%: Slowly pool es1034 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13276 and previous config saved to /var/cache/conftool/dbconfig/20201117-064716-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T07:00:26Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1015 (re)pooling @ 50%: Slowly pool es1015 after cloning es1033 T261717', diff saved to https://phabricator.wikimedia.org/P13277 and previous config saved to /var/cache/conftool/dbconfig/20201117-070025-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T07:00:50Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1019 (re)pooling @ 50%: Slowly pool es1019 after cloning es1034 T261717', diff saved to https://phabricator.wikimedia.org/P13278 and previous config saved to /var/cache/conftool/dbconfig/20201117-070050-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T07:02:09Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1033 (re)pooling @ 20%: Slowly pool es1033 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13280 and previous config saved to /var/cache/conftool/dbconfig/20201117-070209-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T07:02:20Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1034 (re)pooling @ 20%: Slowly pool es1034 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13281 and previous config saved to /var/cache/conftool/dbconfig/20201117-070220-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T07:15:29Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1015 (re)pooling @ 75%: Slowly pool es1015 after cloning es1033 T261717', diff saved to https://phabricator.wikimedia.org/P13282 and previous config saved to /var/cache/conftool/dbconfig/20201117-071529-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T07:15:54Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1019 (re)pooling @ 75%: Slowly pool es1019 after cloning es1034 T261717', diff saved to https://phabricator.wikimedia.org/P13283 and previous config saved to /var/cache/conftool/dbconfig/20201117-071553-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T07:17:13Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1033 (re)pooling @ 25%: Slowly pool es1033 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13284 and previous config saved to /var/cache/conftool/dbconfig/20201117-071712-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T07:17:24Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1034 (re)pooling @ 25%: Slowly pool es1034 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13285 and previous config saved to /var/cache/conftool/dbconfig/20201117-071723-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T07:30:33Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1015 (re)pooling @ 100%: Slowly pool es1015 after cloning es1033 T261717', diff saved to https://phabricator.wikimedia.org/P13286 and previous config saved to /var/cache/conftool/dbconfig/20201117-073032-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T07:30:57Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1019 (re)pooling @ 100%: Slowly pool es1019 after cloning es1034 T261717', diff saved to https://phabricator.wikimedia.org/P13287 and previous config saved to /var/cache/conftool/dbconfig/20201117-073057-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T07:32:16Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1033 (re)pooling @ 30%: Slowly pool es1033 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13288 and previous config saved to /var/cache/conftool/dbconfig/20201117-073216-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T07:32:27Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1034 (re)pooling @ 30%: Slowly pool es1034 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13289 and previous config saved to /var/cache/conftool/dbconfig/20201117-073227-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T07:47:19Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1033 (re)pooling @ 50%: Slowly pool es1033 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13290 and previous config saved to /var/cache/conftool/dbconfig/20201117-074719-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T07:47:31Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1034 (re)pooling @ 50%: Slowly pool es1034 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13291 and previous config saved to /var/cache/conftool/dbconfig/20201117-074730-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T08:02:23Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1033 (re)pooling @ 60%: Slowly pool es1033 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13292 and previous config saved to /var/cache/conftool/dbconfig/20201117-080222-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T08:02:34Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1034 (re)pooling @ 60%: Slowly pool es1034 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13293 and previous config saved to /var/cache/conftool/dbconfig/20201117-080234-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T08:17:26Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1033 (re)pooling @ 75%: Slowly pool es1033 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13294 and previous config saved to /var/cache/conftool/dbconfig/20201117-081726-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T08:17:38Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1034 (re)pooling @ 75%: Slowly pool es1034 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13295 and previous config saved to /var/cache/conftool/dbconfig/20201117-081737-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T08:32:30Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1033 (re)pooling @ 80%: Slowly pool es1033 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13296 and previous config saved to /var/cache/conftool/dbconfig/20201117-083229-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T08:32:41Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1034 (re)pooling @ 80%: Slowly pool es1034 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13297 and previous config saved to /var/cache/conftool/dbconfig/20201117-083241-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T08:47:33Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1033 (re)pooling @ 100%: Slowly pool es1033 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13298 and previous config saved to /var/cache/conftool/dbconfig/20201117-084733-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-17T08:47:44Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1034 (re)pooling @ 100%: Slowly pool es1034 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13299 and previous config saved to /var/cache/conftool/dbconfig/20201117-084744-root.json

The following hosts have been fully pooled in production:

es1033 es2
es1034 es3

Mentioned in SAL (#wikimedia-operations) [2020-11-18T05:45:42Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1018 (re)pooling @ 10%: Slowly pool es1018 after cloning es1032 T261717', diff saved to https://phabricator.wikimedia.org/P13306 and previous config saved to /var/cache/conftool/dbconfig/20201118-054542-root.json

Change 641616 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] instances.yaml: Add es1032

https://gerrit.wikimedia.org/r/641616

Change 641616 merged by Marostegui:
[operations/puppet@production] instances.yaml: Add es1032

https://gerrit.wikimedia.org/r/641616

Change 641617 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] es1032: Enable notifications

https://gerrit.wikimedia.org/r/641617

Mentioned in SAL (#wikimedia-operations) [2020-11-18T06:00:46Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1018 (re)pooling @ 25%: Slowly pool es1018 after cloning es1032 T261717', diff saved to https://phabricator.wikimedia.org/P13307 and previous config saved to /var/cache/conftool/dbconfig/20201118-060045-root.json

Change 641617 merged by Marostegui:
[operations/puppet@production] es1032: Enable notifications

https://gerrit.wikimedia.org/r/641617

Mentioned in SAL (#wikimedia-operations) [2020-11-18T06:06:42Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Pool es1032 with minimum weight on es1 T261717', diff saved to https://phabricator.wikimedia.org/P13308 and previous config saved to /var/cache/conftool/dbconfig/20201118-060641-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2020-11-18T06:15:49Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1018 (re)pooling @ 50%: Slowly pool es1018 after cloning es1032 T261717', diff saved to https://phabricator.wikimedia.org/P13312 and previous config saved to /var/cache/conftool/dbconfig/20201118-061549-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-18T06:25:53Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1032 (re)pooling @ 10%: Slowly pool es1032 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13314 and previous config saved to /var/cache/conftool/dbconfig/20201118-062551-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-18T06:30:53Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1018 (re)pooling @ 75%: Slowly pool es1018 after cloning es1032 T261717', diff saved to https://phabricator.wikimedia.org/P13315 and previous config saved to /var/cache/conftool/dbconfig/20201118-063052-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-18T06:40:55Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1032 (re)pooling @ 20%: Slowly pool es1032 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13316 and previous config saved to /var/cache/conftool/dbconfig/20201118-064054-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-18T06:45:56Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1018 (re)pooling @ 100%: Slowly pool es1018 after cloning es1032 T261717', diff saved to https://phabricator.wikimedia.org/P13317 and previous config saved to /var/cache/conftool/dbconfig/20201118-064556-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-18T06:55:58Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1032 (re)pooling @ 25%: Slowly pool es1032 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13318 and previous config saved to /var/cache/conftool/dbconfig/20201118-065558-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-18T07:11:02Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1032 (re)pooling @ 30%: Slowly pool es1032 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13319 and previous config saved to /var/cache/conftool/dbconfig/20201118-071101-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-18T07:26:05Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1032 (re)pooling @ 50%: Slowly pool es1032 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13320 and previous config saved to /var/cache/conftool/dbconfig/20201118-072605-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-18T07:41:09Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1032 (re)pooling @ 60%: Slowly pool es1032 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13321 and previous config saved to /var/cache/conftool/dbconfig/20201118-074108-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-18T07:56:12Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1032 (re)pooling @ 75%: Slowly pool es1032 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13322 and previous config saved to /var/cache/conftool/dbconfig/20201118-075612-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-18T08:11:15Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1032 (re)pooling @ 80%: Slowly pool es1032 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13323 and previous config saved to /var/cache/conftool/dbconfig/20201118-081115-root.json

Mentioned in SAL (#wikimedia-operations) [2020-11-18T08:26:19Z] <marostegui@cumin1001> dbctl commit (dc=all): 'es1032 (re)pooling @ 100%: Slowly pool es1032 after being recloned T261717', diff saved to https://phabricator.wikimedia.org/P13324 and previous config saved to /var/cache/conftool/dbconfig/20201118-082618-root.json

Marostegui updated the task description. (Show Details)

Next step is to decommission the old hosts, but that will be handled separately on their own tickets

Change 641876 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] install_server: Do not reimage es1032

https://gerrit.wikimedia.org/r/641876

Change 641876 merged by Marostegui:
[operations/puppet@production] install_server: Do not reimage es1032

https://gerrit.wikimedia.org/r/641876