We've migrated on s1 and one s8 host to 10.6
Let's migrate one s7 host to mariadb 10.6 to see if there are issues with any centralauth query. We also need to test the query at T307501: Adjust the field type of globalblocks timestamp columns to fixed binary on wmf wikis
Migrated hosts: db1127, db2122
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | BUG REPORT | Marostegui | T307647 2022-05-05 Wikimedia full site outage | ||
| Stalled | Marostegui | T307501 Adjust the field type of globalblocks timestamp columns to fixed binary on wmf wikis | |||
| Resolved | Marostegui | T301879 Test MariaDB 10.6 on Bullseye | |||
| Resolved | Marostegui | T308126 Migrate a s7 DB host to mariadb 10.6 |
Change 791230 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/puppet@production] db2122: Disable notifications
Change 791230 merged by Marostegui:
[operations/puppet@production] db2122: Disable notifications
Migrated db2122 to 10.6 for (T307501) and I am also going to migrate an eqiad host to tests reads.
Change 791231 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/puppet@production] db2122: Migrate to 10.6
Change 791231 merged by Marostegui:
[operations/puppet@production] db2122: Migrate to 10.6
Mentioned in SAL (#wikimedia-operations) [2022-05-12T05:59:19Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1127 T308126', diff saved to https://phabricator.wikimedia.org/P27794 and previous config saved to /var/cache/conftool/dbconfig/20220512-055918-marostegui.json
Change 791232 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/puppet@production] db1127: Disable notifications
Change 791232 merged by Marostegui:
[operations/puppet@production] db1127: Disable notifications
Change 791234 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/puppet@production] db1127: Migrate to 10.6
Change 791234 merged by Marostegui:
[operations/puppet@production] db1127: Migrate to 10.6
Mentioned in SAL (#wikimedia-operations) [2022-05-12T06:13:05Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Pool db1127 with low weight T308126', diff saved to https://phabricator.wikimedia.org/P27795 and previous config saved to /var/cache/conftool/dbconfig/20220512-061305-marostegui.json
Starting to slowly pool db1127 with very low weight so I can gather slow queries to either wikis or centralauth
Change 791291 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/puppet@production] db1127: Enable notifications
Change 791291 merged by Marostegui:
[operations/puppet@production] db1127: Enable notifications
Enabled slow query log on db1127 (and set it to 10 seconds) so I can start capturing queries
Mentioned in SAL (#wikimedia-operations) [2022-05-12T06:22:42Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Increase traffic on db1127 to test 10.6 T308126', diff saved to https://phabricator.wikimedia.org/P27796 and previous config saved to /var/cache/conftool/dbconfig/20220512-062241-marostegui.json
Mentioned in SAL (#wikimedia-operations) [2022-05-12T06:32:17Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Increase traffic on db1127 to test 10.6 T308126', diff saved to https://phabricator.wikimedia.org/P27797 and previous config saved to /var/cache/conftool/dbconfig/20220512-063217-marostegui.json
For now, the only slow query captured is the one already reported at T305427: Slow recentchanges DB queries on 10.6 on metawiki
Mentioned in SAL (#wikimedia-operations) [2022-05-12T07:57:04Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Increase traffic on db1127 to test 10.6 T308126', diff saved to https://phabricator.wikimedia.org/P27798 and previous config saved to /var/cache/conftool/dbconfig/20220512-075703-marostegui.json
Mentioned in SAL (#wikimedia-operations) [2022-05-12T08:18:14Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Increase traffic on db1127 to test 10.6 T308126', diff saved to https://phabricator.wikimedia.org/P27799 and previous config saved to /var/cache/conftool/dbconfig/20220512-081814-marostegui.json
Mentioned in SAL (#wikimedia-operations) [2022-05-12T09:17:07Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Increase traffic on db1127 to test 10.6 T308126', diff saved to https://phabricator.wikimedia.org/P27800 and previous config saved to /var/cache/conftool/dbconfig/20220512-091706-marostegui.json
Mentioned in SAL (#wikimedia-operations) [2022-05-12T09:46:43Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Increase traffic on db1127 to test 10.6 T308126', diff saved to https://phabricator.wikimedia.org/P27802 and previous config saved to /var/cache/conftool/dbconfig/20220512-094642-marostegui.json
Mentioned in SAL (#wikimedia-operations) [2022-05-12T10:33:33Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Increase traffic on db1127 to test 10.6 T308126', diff saved to https://phabricator.wikimedia.org/P27803 and previous config saved to /var/cache/conftool/dbconfig/20220512-103333-marostegui.json
Mentioned in SAL (#wikimedia-operations) [2022-05-12T10:54:33Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Increase traffic on db1127 to test 10.6 T308126', diff saved to https://phabricator.wikimedia.org/P27804 and previous config saved to /var/cache/conftool/dbconfig/20220512-105432-marostegui.json
Mentioned in SAL (#wikimedia-operations) [2022-05-12T11:54:45Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Increase traffic on db1127 to test 10.6 T308126', diff saved to https://phabricator.wikimedia.org/P27805 and previous config saved to /var/cache/conftool/dbconfig/20220512-115445-marostegui.json
Mentioned in SAL (#wikimedia-operations) [2022-05-12T12:27:07Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Increase traffic on db1127 to test 10.6 T308126', diff saved to https://phabricator.wikimedia.org/P27806 and previous config saved to /var/cache/conftool/dbconfig/20220512-122707-marostegui.json
db1127 is now fully repooled with its original weight. Monitoring how it goes in terms of slow queries, especially those against centralauth
db1127 looking good, no slow queries reported on centralauth. I think I am going to leave this host serving during the weekend as well.