Page MenuHomePhabricator

Switchover es4 codfw master es2020 -> es2021
Closed, ResolvedPublic

Description

When: Anytime, writes will be disabled

Checklist:

NEW primary: es2021
OLD primary: es2020

  • Check configuration differences between new and old primary:
sudo pt-config-diff --defaults-file /root/.my.cnf h=es2020.codfw.wmnet h=es2021.codfw.wmnet

Failover prep:

  • Silence alerts on all hosts:
sudo cookbook sre.hosts.downtime --hours 1 -r "Primary switchover es4 T356064" 'A:db-section-es4'
  • Set NEW primary with weight 0 (and depool it from API or vslow/dump groups if it is present).
sudo dbctl instance es2021 set-weight 0
sudo dbctl config commit -m "Set es2021 with weight 0 T356064"
  • Topology changes, move all replicas under NEW primary
sudo db-switchover --timeout=25 --only-slave-move es2020 es2021
  • Disable puppet on both nodes
sudo cumin 'es2020* or es2021*' 'disable-puppet "primary switchover T356064"'

Failover:

  • Log the failover:
!log Starting es4 codfw failover from es2020 to es2021 - T356064
  • Switch primaries:
sudo db-switchover --skip-slave-move es2020 es2021
echo "===== es1020 (OLD)"; sudo db-mysql es2020 -e 'show slave status\G'
echo "===== es1021 (NEW)"; sudo db-mysql es2021 -e 'show slave status\G'
  • Promote NEW primary in dbctl, and remove read-only
sudo dbctl --scope codfw section es4 set-master es2021
sudo dbctl config commit -m "Promote es2021 to es4 primary T356064"
  • Restart puppet on both hosts:
sudo cumin 'es2020* or es2021*' 'run-puppet-agent -e "primary switchover T356064"'

Clean up tasks:

  • Clean up heartbeat table(s).
sudo db-mysql es2021 heartbeat -e "delete from heartbeat where file like 'es2020%';"
  • change events for query killer:
events_coredb_master.sql on the new primary es2021
events_coredb_slave.sql on the new slave es2020
sudo dbctl instance es2020 set-candidate-master --section es4 true
sudo dbctl instance es2021 set-candidate-master --section es4 false
(dborch1001): sudo orchestrator-client -c untag -i es2021 --tag name=candidate
(dborch1001): sudo orchestrator-client -c tag -i es2020 --tag name=candidate
sudo db-mysql db1215 zarcillo -e "select * from masters where section = 'es4';"
  • (If needed): Depool es2020 for maintenance.
sudo dbctl instance es2020 depool
sudo dbctl config commit -m "Depool es2020 T356064"
  • Change es2020 weight to mimic the previous weight es2021:
sudo dbctl instance es2020 edit
  • Update/resolve this ticket.

Event Timeline

Marostegui updated the task description. (Show Details)

Change 993711 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/mediawiki-config@master] db-production.php: Disable writes on es4

https://gerrit.wikimedia.org/r/993711

Mentioned in SAL (#wikimedia-operations) [2024-01-30T06:18:21Z] <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 1:00:00 on 6 hosts with reason: Primary switchover es4 T356064

Mentioned in SAL (#wikimedia-operations) [2024-01-30T06:18:39Z] <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 6 hosts with reason: Primary switchover es4 T356064

Change 993711 merged by jenkins-bot:

[operations/mediawiki-config@master] db-production.php: Disable writes on es4

https://gerrit.wikimedia.org/r/993711

Change 993835 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/puppet@production] es4: Promote es2021 to master

https://gerrit.wikimedia.org/r/993835

Mentioned in SAL (#wikimedia-operations) [2024-01-30T06:21:18Z] <marostegui@deploy2002> Started scap: Backport for [[gerrit:993711|db-production.php: Disable writes on es4 (T356064)]]

Change 993836 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/dns@master] wmnet: Update es4 master

https://gerrit.wikimedia.org/r/993836

Mentioned in SAL (#wikimedia-operations) [2024-01-30T06:22:25Z] <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 1:00:00 on 6 hosts with reason: Primary switchover es4 T356064

Mentioned in SAL (#wikimedia-operations) [2024-01-30T06:22:32Z] <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 6 hosts with reason: Primary switchover es4 T356064

Mentioned in SAL (#wikimedia-operations) [2024-01-30T06:22:42Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Set es2020 with weight 0 T356064', diff saved to https://phabricator.wikimedia.org/P55856 and previous config saved to /var/cache/conftool/dbconfig/20240130-062241-marostegui.json

codfw/externalLoads/es4 live                                       codfw/externalLoads/es4 generated
[                                                                  [
    {                                                                  {
        "es2020": 20                                                       "es2020": 0
    },                                                                 },
    {                                                                  {
        "es2021": 100,                                                     "es2021": 100,
        "es2022": 100                                                      "es2022": 100
    }                                                                  }

Mentioned in SAL (#wikimedia-operations) [2024-01-30T06:22:55Z] <marostegui@deploy2002> marostegui: Backport for [[gerrit:993711|db-production.php: Disable writes on es4 (T356064)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Change 993835 merged by Marostegui:

[operations/puppet@production] es4: Promote es2021 to master

https://gerrit.wikimedia.org/r/993835

Mentioned in SAL (#wikimedia-operations) [2024-01-30T06:30:29Z] <marostegui@deploy2002> Finished scap: Backport for [[gerrit:993711|db-production.php: Disable writes on es4 (T356064)]] (duration: 09m 11s)

Mentioned in SAL (#wikimedia-operations) [2024-01-30T06:35:57Z] <marostegui> Starting es4 codfw failover from es2020 to es2021 - T356064

Mentioned in SAL (#wikimedia-operations) [2024-01-30T06:36:26Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Promote es2021 to es4 primary T356064', diff saved to https://phabricator.wikimedia.org/P55860 and previous config saved to /var/cache/conftool/dbconfig/20240130-063625-root.json

Change 993836 merged by Marostegui:

[operations/dns@master] wmnet: Update es4 master

https://gerrit.wikimedia.org/r/993836

Mentioned in SAL (#wikimedia-operations) [2024-01-30T06:45:12Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Reduce es2021 weight T356064', diff saved to https://phabricator.wikimedia.org/P55862 and previous config saved to /var/cache/conftool/dbconfig/20240130-064512-root.json

Mentioned in SAL (#wikimedia-operations) [2024-01-30T06:45:26Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es2020 T356064', diff saved to https://phabricator.wikimedia.org/P55863 and previous config saved to /var/cache/conftool/dbconfig/20240130-064526-root.json

Change 993838 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/puppet@production] es2020: Migrate to mariadb 10.6

https://gerrit.wikimedia.org/r/993838

Change 993838 merged by Marostegui:

[operations/puppet@production] es2020: Migrate to mariadb 10.6

https://gerrit.wikimedia.org/r/993838

Marostegui updated the task description. (Show Details)

Writes enabled and on binlogs.