After a (no) decision was taken at T193224, tracking the progress in an epic task of our progress of upgrading all databases, and services around them (monitoring, backups, etc.) to Debian Buster and MariaDB 10.4.
MariaDB 10.1 EOL 17th Oct 2020: https://mariadb.com/wp-content/uploads/2019/07/mariadb-engineering-policies-v2-01_policy_1036.pdf
Script wmf-auto-reimage was launched by marostegui on cumin2001.codfw.wmnet for hosts:
['db1094.eqiad.wmnet']
The log can be found in /var/log/wmf-auto-reimage/202006220943_marostegui_29419.log.
Change 607020 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] multiinstance.pp: Change the mariadb basedir depending on the OS
Change 607020 merged by Marostegui:
[operations/puppet@production] multiinstance.pp: Change the mariadb basedir depending on the OS
Change 607034 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] core/multiinstance.pp: Change the mariadb basedir depending on the OS
Change 607034 merged by Marostegui:
[operations/puppet@production] core/multiinstance.pp: Change the mariadb basedir depending on the OS
Change 607223 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db2133: Reimage to Buster
Change 607223 merged by Marostegui:
[operations/puppet@production] db2133: Reimage to Buster
Mentioned in SAL (#wikimedia-operations) [2020-06-23T07:30:42Z] <marostegui> Reimage db2133 (m2 codfw master) to Buster (this will trigger haproxy IRC alert) T250666
Script wmf-auto-reimage was launched by marostegui on cumin2001.codfw.wmnet for hosts:
['db2133.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/202006230731_marostegui_9331.log.
Change 607260 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/puppet@production] install_server: Switch db1088 to buster.
Change 607260 merged by Kormat:
[operations/puppet@production] install_server: Switch db1088 to buster.
Mentioned in SAL (#wikimedia-operations) [2020-06-23T10:22:46Z] <kormat> reimaging db1088 to buster T250666
Script wmf-auto-reimage was launched by kormat on cumin1001.eqiad.wmnet for hosts:
['db1088.eqiad.wmnet']
The log can be found in /var/log/wmf-auto-reimage/202006231025_kormat_31998.log.
Change 607447 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] install_server: Reimage db2120 to Buster
Change 607447 merged by Marostegui:
[operations/puppet@production] install_server: Reimage db2120 to Buster
Script wmf-auto-reimage was launched by marostegui on cumin2001.codfw.wmnet for hosts:
['db2120.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/202006240854_marostegui_7222.log.
Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:
['db2081.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/202007160810_marostegui_2449.log.
Change 616989 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db2106: Reimage to Buster
Change 616989 merged by Marostegui:
[operations/puppet@production] db2106: Reimage to Buster
Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:
['db2106.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/202007290527_marostegui_24725.log.
Change 616995 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db2117: Reimage to Buster
Change 616995 merged by Marostegui:
[operations/puppet@production] db2117: Reimage to Buster
Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:
['db2117.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/202007290631_marostegui_20628.log.
Change 617383 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db2083: Reimage to Buster
Change 617383 merged by Marostegui:
[operations/puppet@production] db2083: Reimage to Buster
Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:
['db2083.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/202007300737_marostegui_18826.log.
Change 618172 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] mariadb: Reimage db1119 to Buster
Change 618172 merged by Marostegui:
[operations/puppet@production] mariadb: Reimage db1119 to Buster
Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:
['db1119.eqiad.wmnet']
The log can be found in /var/log/wmf-auto-reimage/202008040524_marostegui_8632.log.
Change 618921 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db1092: Reimage to Buster
Change 618921 merged by Marostegui:
[operations/puppet@production] db1092: Reimage to Buster
Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:
['db1092.eqiad.wmnet']
The log can be found in /var/log/wmf-auto-reimage/202008070639_marostegui_4653.log.
Change 619698 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] mariadb: Reimage db1110 to Buster
Change 619698 merged by Marostegui:
[operations/puppet@production] mariadb: Reimage db1110 to Buster
Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:
['db1110.eqiad.wmnet']
The log can be found in /var/log/wmf-auto-reimage/202008120736_marostegui_3997.log.
Change 620528 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] install_server: Reimage db1099 to Buster
Change 620528 merged by Marostegui:
[operations/puppet@production] install_server: Reimage db1099 to Buster
Change 625863 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] install_server: Reimage db1133 into buster
Change 625863 merged by Jcrespo:
[operations/puppet@production] install_server: Reimage db1133 into buster
Change 673220 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] dbbackups: Move s5 from db2139 to db2101
Change 673220 merged by Jcrespo:
[operations/puppet@production] dbbackups: Move s5 from db2139 to db2101
@Marostegui @Kormat This is a follow up to our conversation on our last meeting: It doesn't have to happen now, but it would help me to have a sorted list (even if subject to change) of section upgrade order. I know you mention s6 next and maybe s5 or s2 later, and I am already preparing for that. It would help me to reorganize the source backup hosts (and I think to yourself, for your multiinstance hosts) so upgrades are just a configuration switch on my side.
Obviously, this is Q4-kinda request, not needed now. :-) Also may need its own separate ticket, up to you.
The first one will be s6 for sure, and the second one either s5 or s2. I am more inclined for s5, but subject to change.
Mentioned in SAL (#wikimedia-operations) [2021-03-30T08:36:22Z] <jynus> mariadb upgrade of all buster source backup hosts to 10.4.18 T250666
This is all done apart from db1115 which is running Stretch until we get rid of Tendril, which hopefully will happen in the next few months T297605: Shutdown Tendril and dbtree