pc2 have been running Buster and 10.4 for a few weeks now and the spare hosts (pc1010 and pc2010) have just been upgraded.
Let's upgrade pc1 and pc3 also so we can have the full pc infra running Buster and 10.4
This involves MW depooling.
My suggestion would be to upgrade:
pc1: first codfw, then eqiad
pc3: first codfw, then eqiad
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Stalled | None | T302086 Set scap minimum python version to 3.7 | |||
| Resolved | None | T247045 Migrate all of production metal and VMs to Buster or later | |||
| Resolved | Marostegui | T250666 Upgrade WMF database-and-backup-related hosts to buster | |||
| Resolved | • Kormat | T252182 Upgrade parsercache to Buster and MariaDB 10.4 |
Assigning to @Kormat as I think it can be a good exercise as it involves MW deployments.
Mentioned in SAL (#wikimedia-operations) [2020-05-12T10:43:52Z] <kormat> reimaging pc2010 to buster T252182
Change 595901 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/puppet@production] install_server: Allow reimage of pc2010
Change 595901 merged by Kormat:
[operations/puppet@production] install_server: Allow reimage of pc2010
Script wmf-auto-reimage was launched by kormat on cumin1001.eqiad.wmnet for hosts:
['pc2010.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/202005121059_kormat_190456.log.
Change 595922 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/puppet@production] install_server: Allow reimage of pc2007
Change 595922 merged by Kormat:
[operations/puppet@production] install_server: Allow reimage of pc2007
Change 595928 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/puppet@production] install_server: Switch all remaining pc* hosts to buster.
Change 595928 merged by Kormat:
[operations/puppet@production] install_server: Switch all remaining pc* hosts to buster.
Mentioned in SAL (#wikimedia-operations) [2020-05-12T13:36:33Z] <kormat> reimaging pc2007 to buster T252182
Script wmf-auto-reimage was launched by kormat on cumin1001.eqiad.wmnet for hosts:
['pc2007.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/202005121339_kormat_212939.log.
Change 595945 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/puppet@production] install_server: Fix netboot.cfg entry for pc2007
Change 595945 merged by Kormat:
[operations/puppet@production] install_server: Fix netboot.cfg entry for pc2007
Script wmf-auto-reimage was launched by kormat on cumin1001.eqiad.wmnet for hosts:
['pc2007.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/202005121447_kormat_222215.log.
Completed auto-reimage of hosts:
['pc2007.codfw.wmnet']
Of which those FAILED:
['pc2007.codfw.wmnet']
Script wmf-auto-reimage was launched by kormat on cumin1001.eqiad.wmnet for hosts:
['pc2007.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/202005121449_kormat_222380.log.
Change 595961 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/puppet@production] Revert "install_server: Allow reimage of pc2010"
Change 595961 merged by Kormat:
[operations/puppet@production] Revert "install_server: Allow reimage of pc2010"
Change 596146 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/puppet@production] install_server: Disallow reimaging of pc2007
Change 596146 merged by Kormat:
[operations/puppet@production] install_server: Disallow reimaging of pc2007
Change 596152 had a related patch set uploaded (by Kormat; owner: Stephen Shirley):
[operations/mediawiki-config@master] db-eqiad.php: Pool pc1010 for pc1
Change 596152 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Pool pc1010 for pc1
Mentioned in SAL (#wikimedia-operations) [2020-05-13T08:52:09Z] <kormat@deploy1001> Synchronized wmf-config/db-eqiad.php: Pool pc1010 as pc1 master T252182 (duration: 01m 17s)
Change 596158 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/puppet@production] install_server: Allow reimage of pc1007
Change 596158 merged by Kormat:
[operations/puppet@production] install_server: Allow reimage of pc1007
Script wmf-auto-reimage was launched by kormat on cumin1001.eqiad.wmnet for hosts:
['pc1007.eqiad.wmnet']
The log can be found in /var/log/wmf-auto-reimage/202005130924_kormat_89303.log.
Change 596173 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/mediawiki-config@master] Revert "db-eqiad.php: Pool pc1010 for pc1"
Change 596173 merged by jenkins-bot:
[operations/mediawiki-config@master] Revert "db-eqiad.php: Pool pc1010 for pc1"
Mentioned in SAL (#wikimedia-operations) [2020-05-13T10:09:03Z] <kormat@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool pc1007 as pc1 master T252182 (duration: 01m 05s)
Change 596176 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/puppet@production] Revert "install_server: Allow reimage of pc1007"
Change 596176 merged by Kormat:
[operations/puppet@production] Revert "install_server: Allow reimage of pc1007"
Mentioned in SAL (#wikimedia-operations) [2020-06-03T08:08:44Z] <kormat> reimaging pc2009 to buster T252182
Change 602014 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/puppet@production] install_server: Allow reimage of pc2009
Change 602019 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/mediawiki-config@master] db-codfw.php: Replace pc2009 with pc2010 while reimaging
Change 602014 merged by Kormat:
[operations/puppet@production] install_server: Allow reimage of pc2009
Change 602019 merged by jenkins-bot:
[operations/mediawiki-config@master] db-codfw.php: Replace pc2009 with pc2010 while reimaging
Script wmf-auto-reimage was launched by kormat on cumin1001.eqiad.wmnet for hosts:
['pc2009.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/202006030906_kormat_72120.log.
Change 602038 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/puppet@production] mariadb: Enable notifications for db2009
Change 602038 merged by Kormat:
[operations/puppet@production] mariadb: Enable notifications for db2009
Mentioned in SAL (#wikimedia-operations) [2020-06-03T13:13:36Z] <kormat@deploy1001> Synchronized wmf-config/db-codfw.php: Put pc2009 back into pc3 after reimaging T252182 (duration: 01m 05s)
Change 602081 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/puppet@production] mariadb: Allow reimage of pc1009
Mentioned in SAL (#wikimedia-operations) [2020-06-03T13:44:07Z] <kormat> reimaging pc1007 to buster, wish me luck T252182
Change 602081 merged by Kormat:
[operations/puppet@production] mariadb: Allow reimage of pc1009
Mentioned in SAL (#wikimedia-operations) [2020-06-03T13:47:34Z] <kormat> reimaging *pc1009 (promise) to buster T252182
Mentioned in SAL (#wikimedia-operations) [2020-06-03T13:59:56Z] <kormat@deploy1001> Synchronized wmf-config/db-eqiad.php: Replace pc1009 with pc1010 reimaging T252182 (duration: 01m 06s)
Script wmf-auto-reimage was launched by kormat on cumin1001.eqiad.wmnet for hosts:
['pc1009.eqiad.wmnet']
The log can be found in /var/log/wmf-auto-reimage/202006031405_kormat_68270.log.
Mentioned in SAL (#wikimedia-operations) [2020-06-03T14:50:40Z] <kormat@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool pc1009 in pc3 after reimaging T252182 (duration: 01m 06s)