After gerrit:1119745 has been merged and landed, we should set up ms1, ms2, and ms3 clusters similar to ParserCache. We should keep the dbctl config for x2 until we switch the mw config to use ms clusters.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Ladsgroup | T383327 Re-architecture mainstash (x2) to allow easier maintenance | |||
| Resolved | • Marostegui | T387332 Set up ms1, ms2, ms3 db clusters |
Change #1122945 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/puppet@production] valid_sections.pp: Add ms1, ms2, and ms3
Change #1122945 merged by Marostegui:
[operations/puppet@production] valid_sections.pp: Add ms1, ms2, and ms3
Change #1123181 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/puppet@production] section.yaml: Add ms1, ms2, ms3
Change #1123273 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/puppet@production] mariadb: Move db1153, db2143 to ms3
Change #1123181 merged by Marostegui:
[operations/puppet@production] section.yaml: Add ms1, ms2, ms3
Change #1123273 merged by Marostegui:
[operations/puppet@production] mariadb: Move db1153, db2143 to ms3
Change #1123587 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/puppet@production] instance.schema: Add ms1,ms2 and ms3
Change #1123587 merged by Marostegui:
[operations/puppet@production] instance.schema: Add ms1,ms2 and ms3
Change #1123589 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/puppet@production] dbconfig.schema: Add ms1,ms2,ms3
Change #1123589 merged by Marostegui:
[operations/puppet@production] dbconfig.schema: Add ms1,ms2,ms3
Mentioned in SAL (#wikimedia-operations) [2025-02-28T09:08:39Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Add ms3 T387332', diff saved to https://phabricator.wikimedia.org/P73841 and previous config saved to /var/cache/conftool/dbconfig/20250228-090838-marostegui.json
Change #1123596 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/dns@master] wmnet: Add ms3-master CNAME
Change #1123596 merged by Marostegui:
[operations/dns@master] wmnet: Add ms3-master CNAME
@Ladsgroup I've set up ms3.
I will go for ms2 in a bit, the question is, ms1 will be basically renaming x2, is MW ready for x2 being gone and being converted to ms1?
Not before the mw-config patch being deployed, I suggest in dbctl keeping both x2 and ms1, deploy the patch, make sure it's working fine and then remove x2 from dbctl (then a mw-config patch needed to remove x2 stuff too).
sorry, a mw-patch is needed after deploying mw patch that starts using msX sections to just ignore x2 to avoid issues we had like what we had in the morning. So it'll be something like this:
That works then! x2 and ms1 will have shared hosts, but that should be perfectly okay from MW and dbctl point of view.
Change #1125993 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/puppet@production] mariadb: Set up ms1
Mentioned in SAL (#wikimedia-operations) [2025-03-10T11:17:43Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Push ms1 config T387332', diff saved to https://phabricator.wikimedia.org/P74169 and previous config saved to /var/cache/conftool/dbconfig/20250310-111742-marostegui.json
Change #1125993 merged by Marostegui:
[operations/puppet@production] mariadb: Set up ms1
Mentioned in SAL (#wikimedia-operations) [2025-03-10T11:20:47Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Add weight to ms1 hosts T387332', diff saved to https://phabricator.wikimedia.org/P74170 and previous config saved to /var/cache/conftool/dbconfig/20250310-112046-marostegui.json
Change #1125995 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/puppet@production] site: Add a note about ms1
Change #1125995 merged by Marostegui:
[operations/puppet@production] site: Add a note about ms1
Change #1126028 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/puppet@production] mariadb: Add ms1,ms2 and ms3
Change #1126028 merged by Marostegui:
[operations/puppet@production] mariadb: Add ms1,ms2 and ms3
Change #1130600 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/puppet@production] mariadb: Move hosts to ms3
Change #1130600 merged by Marostegui:
[operations/puppet@production] mariadb: Move hosts to ms2
Change #1130639 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/puppet@production] db1151,db2144: Make them masters in ms2
Change #1130639 merged by Marostegui:
[operations/puppet@production] db1151,db2144: Make them masters in ms2
Mentioned in SAL (#wikimedia-operations) [2025-03-24T15:56:16Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Add ms2 to live traffic T387332', diff saved to https://phabricator.wikimedia.org/P74354 and previous config saved to /var/cache/conftool/dbconfig/20250324-155616-marostegui.json
Change #1130882 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/puppet@production] db1151,db2144: Enable notifications
Change #1130882 merged by Marostegui:
[operations/puppet@production] db1151,db2144: Enable notifications
Mentioned in SAL (#wikimedia-operations) [2025-03-25T09:58:17Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Depool ms1 T387332', diff saved to https://phabricator.wikimedia.org/P74372 and previous config saved to /var/cache/conftool/dbconfig/20250325-095817-marostegui.json
Mentioned in SAL (#wikimedia-operations) [2025-03-25T10:11:02Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repool ms1 T387332', diff saved to https://phabricator.wikimedia.org/P74375 and previous config saved to /var/cache/conftool/dbconfig/20250325-101101-marostegui.json
Mentioned in SAL (#wikimedia-operations) [2025-03-25T10:18:06Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Remove hosts from x2 T387332', diff saved to https://phabricator.wikimedia.org/P74377 and previous config saved to /var/cache/conftool/dbconfig/20250325-101805-marostegui.json
Change #1130970 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/puppet@production] conftool: Remove x2 from allowed sections
Change #1130970 merged by Marostegui:
[operations/puppet@production] conftool: Remove x2 from allowed sections
Change #1130972 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/puppet@production] sections.yaml: Remove x2
Change #1130972 merged by Marostegui:
[operations/puppet@production] sections.yaml: Remove x2
Change #1130974 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/puppet@production] valid_section.pp: Remove x2
Change #1130974 merged by Marostegui:
[operations/puppet@production] valid_section.pp: Remove x2
Change #1131131 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/puppet@production] db1152,db2142: Disable sync_binlog
Change #1131131 merged by Marostegui:
[operations/puppet@production] db1152,db2142: Disable sync_binlog
Change #1131888 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/software@master] check_depooled: Remove x2, add ms[123]
Change #1131888 merged by jenkins-bot:
[operations/software@master] check_depooled: Remove x2, add ms[123]
Change #1134928 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/puppet@production] production-ms.sql.erb: Add file
Change #1134928 merged by Marostegui:
[operations/puppet@production] production-ms.sql.erb: Add file
Change #1135385 had a related patch set uploaded (by Alexandros Kosiaris; author: Alexandros Kosiaris):
[operations/puppet@production] Remove x2, add ms{1,2,3} to profile::mariadb::section_ports:
Change #1135385 merged by Alexandros Kosiaris:
[operations/puppet@production] Remove x2, add ms{1,2,3} to profile::mariadb::section_ports: