Page MenuHomePhabricator

Install 1 buster+10.4 host per section
Closed, ResolvedPublic

Description

  • s1
    • db1107
    • db2085:3311
  • s2
    • db1103:3312
    • db2125
  • s3
    • db1078
    • db2109
  • s4
    • db1103:3314
    • db2084:3314
  • s5
    • db1096:3315
    • db2084:3315
  • s6
    • db1096:3316
    • db1098:3316
    • db2114
  • s7
    • db1098:3317
    • db2121
  • s8
    • db1111
    • db2085:3318
  • x1
    • db1137
    • db2115
  • pc2
    • pc1008
    • pc2008

Details

SubjectRepoBranchLines +/-
operations/puppetproduction+1 -1
operations/puppetproduction+1 -1
operations/puppetproduction+0 -1
operations/puppetproduction+1 -1
operations/puppetproduction+0 -1
operations/puppetproduction+1 -1
operations/puppetproduction+0 -1
operations/puppetproduction+1 -1
operations/puppetproduction+0 -1
operations/puppetproduction+1 -1
operations/puppetproduction+0 -1
operations/puppetproduction+1 -1
operations/puppetproduction+0 -1
operations/puppetproduction+1 -1
operations/puppetproduction+1 -1
operations/puppetproduction+1 -0
operations/puppetproduction+0 -2
operations/puppetproduction+1 -1
operations/puppetproduction+0 -2
operations/puppetproduction+0 -1
operations/puppetproduction+1 -2
operations/puppetproduction+1 -1
operations/puppetproduction+1 -0
Show related patches Customize query in gerrit

Related Objects

StatusSubtypeAssignedTask
StalledNone
ResolvedNone
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
DeclinedMarostegui
ResolvedMarostegui
ResolvedJclark-ctr
ResolvedKormat
ResolvedKormat
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedKormat
ResolvedKormat
ResolvedMarostegui
Resolvedjcrespo
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
Resolvedtaavi
ResolvedNone
Resolvedzeljkofilipin
Resolvedtaavi
ResolvedMarostegui
ResolvedMarostegui
ResolvedCmjohnson
Resolveddcaro
ResolvedMarostegui
ResolvedRequestwiki_willy
ResolvedRequestCmjohnson
ResolvedRequestCmjohnson
ResolvedRequestCmjohnson
ResolvedRequestCmjohnson
ResolvedMoritzMuehlenhoff
ResolvedMarostegui
ResolvedMarostegui
DuplicateNone
ResolvedKormat
ResolvedKormat
Resolvedsgrabarczuk
Resolvedjcrespo
Resolvedjcrespo
Resolvedjcrespo
ResolvedKormat
ResolvedKormat
Resolvedsgrabarczuk
ResolvedKormat

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Mentioned in SAL (#wikimedia-operations) [2020-03-06T07:50:18Z] <marostegui> Stop MySQL on db2085:3311, db2085:3318 for reimage to buster T246604

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db2085.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202003060755_marostegui_257799.log.

Completed auto-reimage of hosts:

['db2085.codfw.wmnet']

and were ALL successful.

Mentioned in SAL (#wikimedia-operations) [2020-03-06T08:25:49Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1078 T246604', diff saved to https://phabricator.wikimedia.org/P10637 and previous config saved to /var/cache/conftool/dbconfig/20200306-082549-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2020-03-06T08:28:59Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db2085:3311, db2085:3318 after reimage to buster - T246604', diff saved to https://phabricator.wikimedia.org/P10638 and previous config saved to /var/cache/conftool/dbconfig/20200306-082858-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2020-03-06T08:41:41Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1078 T246604', diff saved to https://phabricator.wikimedia.org/P10639 and previous config saved to /var/cache/conftool/dbconfig/20200306-084141-marostegui.json

Change 577518 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] install_server: Allow reimage db2084

https://gerrit.wikimedia.org/r/577518

Change 577518 merged by Marostegui:
[operations/puppet@production] install_server: Allow reimage db2084

https://gerrit.wikimedia.org/r/577518

Mentioned in SAL (#wikimedia-operations) [2020-03-06T09:21:04Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db2084:3314, db2084:3315 for reimage to buster - T246604', diff saved to https://phabricator.wikimedia.org/P10645 and previous config saved to /var/cache/conftool/dbconfig/20200306-092103-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2020-03-06T09:21:36Z] <marostegui> Stop MySQL on db2084:3315, db2084:3314 for reimage T246604

Change 577520 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db2084: Reimage to buster

https://gerrit.wikimedia.org/r/577520

Change 577520 merged by Marostegui:
[operations/puppet@production] db2084: Reimage to buster

https://gerrit.wikimedia.org/r/577520

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db2084.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202003060928_marostegui_13048.log.

Completed auto-reimage of hosts:

['db2084.codfw.wmnet']

and were ALL successful.

Mentioned in SAL (#wikimedia-operations) [2020-03-06T09:54:07Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db2084:3314, db2084:3315 after reimage to buster - T246604', diff saved to https://phabricator.wikimedia.org/P10647 and previous config saved to /var/cache/conftool/dbconfig/20200306-095407-marostegui.json

Change 578172 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] install_server: Allow reimage db2114

https://gerrit.wikimedia.org/r/578172

Change 578172 merged by Marostegui:
[operations/puppet@production] install_server: Allow reimage db2114

https://gerrit.wikimedia.org/r/578172

Mentioned in SAL (#wikimedia-operations) [2020-03-09T07:09:38Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db2114 for reimage to buster - T246604', diff saved to https://phabricator.wikimedia.org/P10654 and previous config saved to /var/cache/conftool/dbconfig/20200309-070937-marostegui.json

Change 578178 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] install_server: Reimage db2114 to buster

https://gerrit.wikimedia.org/r/578178

Change 578178 merged by Marostegui:
[operations/puppet@production] install_server: Reimage db2114 to buster

https://gerrit.wikimedia.org/r/578178

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db2114.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202003090715_marostegui_180669.log.

Completed auto-reimage of hosts:

['db2114.codfw.wmnet']

and were ALL successful.

Mentioned in SAL (#wikimedia-operations) [2020-03-09T07:46:30Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db2114 after reimage to buster - T246604', diff saved to https://phabricator.wikimedia.org/P10655 and previous config saved to /var/cache/conftool/dbconfig/20200309-074629-marostegui.json

Change 578280 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] install_server: Allow reimage of db2126

https://gerrit.wikimedia.org/r/578280

Change 578280 merged by Marostegui:
[operations/puppet@production] install_server: Allow reimage of db2126

https://gerrit.wikimedia.org/r/578280

Mentioned in SAL (#wikimedia-operations) [2020-03-09T08:21:19Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db2126 for reimage to buster - T246604', diff saved to https://phabricator.wikimedia.org/P10656 and previous config saved to /var/cache/conftool/dbconfig/20200309-082118-marostegui.json

Change 578281 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] install_server: Reimage db2126 to buster

https://gerrit.wikimedia.org/r/578281

Change 578281 merged by Marostegui:
[operations/puppet@production] install_server: Reimage db2126 to buster

https://gerrit.wikimedia.org/r/578281

db2126 is the sanitarium master for s2, so I am not going to re-image it. Will pick another one for s2

Change 578285 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] install_server: Allow reimage of db2125

https://gerrit.wikimedia.org/r/578285

Mentioned in SAL (#wikimedia-operations) [2020-03-09T08:37:12Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db2125 for reimage to buster - T246604', diff saved to https://phabricator.wikimedia.org/P10658 and previous config saved to /var/cache/conftool/dbconfig/20200309-083711-marostegui.json

Change 578285 merged by Marostegui:
[operations/puppet@production] install_server: Allow reimage of db2125

https://gerrit.wikimedia.org/r/578285

Change 578287 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] install_server: Reimage db2125 to buster

https://gerrit.wikimedia.org/r/578287

Change 578287 merged by Marostegui:
[operations/puppet@production] install_server: Reimage db2125 to buster

https://gerrit.wikimedia.org/r/578287

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db2125.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202003090845_marostegui_196714.log.

Completed auto-reimage of hosts:

['db2125.codfw.wmnet']

and were ALL successful.

Mentioned in SAL (#wikimedia-operations) [2020-03-09T15:24:28Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db2125 - T246604', diff saved to https://phabricator.wikimedia.org/P10666 and previous config saved to /var/cache/conftool/dbconfig/20200309-152427-marostegui.json

Change 578472 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] install_server: Allow reimage db2121

https://gerrit.wikimedia.org/r/578472

Change 578472 merged by Marostegui:
[operations/puppet@production] install_server: Allow reimage db2121

https://gerrit.wikimedia.org/r/578472

Change 578473 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] install_server: Reimage db2121 to buster

https://gerrit.wikimedia.org/r/578473

Change 578473 merged by Marostegui:
[operations/puppet@production] install_server: Reimage db2121 to buster

https://gerrit.wikimedia.org/r/578473

Mentioned in SAL (#wikimedia-operations) [2020-03-10T13:46:48Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db2121 for reimage to buster - T246604', diff saved to https://phabricator.wikimedia.org/P10676 and previous config saved to /var/cache/conftool/dbconfig/20200310-134648-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2020-03-10T13:52:31Z] <marostegui> Stop mysql on db2121 for reimage to buster T246604

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db2121.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202003101355_marostegui_227323.log.

Completed auto-reimage of hosts:

['db2121.codfw.wmnet']

and were ALL successful.

Mentioned in SAL (#wikimedia-operations) [2020-03-10T14:48:17Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db2121 - T246604', diff saved to https://phabricator.wikimedia.org/P10677 and previous config saved to /var/cache/conftool/dbconfig/20200310-144817-root.json

Some of these hosts had events disabled due to T247728
I have checked and enabled the ones missing.

Marostegui updated the task description. (Show Details)

This is done

Change 583211 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] install_server: Allow reimage db1137

https://gerrit.wikimedia.org/r/583211

Change 583211 merged by Marostegui:
[operations/puppet@production] install_server: Allow reimage db1137

https://gerrit.wikimedia.org/r/583211

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db1137.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202003250839_marostegui_81114.log.

Completed auto-reimage of hosts:

['db1137.eqiad.wmnet']

and were ALL successful.

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db2115.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202003251247_marostegui_113904.log.

Completed auto-reimage of hosts:

['db2115.codfw.wmnet']

Of which those FAILED:

['db2115.codfw.wmnet']

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db2115.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202003251336_marostegui_119447.log.

Completed auto-reimage of hosts:

['db2115.codfw.wmnet']

and were ALL successful.

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['pc2008.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202004090539_marostegui_55860.log.

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['pc2008.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202004090547_marostegui_56325.log.

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['pc2008.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202004090626_marostegui_61258.log.

Completed auto-reimage of hosts:

['pc2008.codfw.wmnet']

and were ALL successful.

pc2008 is now running buster and 10.4
pt-heartbeat seems to be also running without issues.

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['pc1008.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202004100603_marostegui_235860.log.

Completed auto-reimage of hosts:

['pc1008.eqiad.wmnet']

and were ALL successful.