Page MenuHomePhabricator

Migrate s8 to 10.11
Closed, ResolvedPublic

Description

See parent

Status:

+----------+----------------------+
| hostname | mariadb_version      |
+----------+----------------------+
| db2198   | 10.11.13-MariaDB-log |
| db2152   | 10.11.14-MariaDB-log |
| db2154   | 10.11.14-MariaDB-log |
| db2161   | 10.11.14-MariaDB-log |
| db2163   | 10.11.14-MariaDB-log |
| db2164   | 10.11.14-MariaDB-log |
| db2166   | 10.11.14-MariaDB-log |
| db2167   | 10.11.14-MariaDB-log |
| db2181   | 10.11.14-MariaDB-log |
| db2195   | 10.11.14-MariaDB-log |
| db2165   | 10.11.14-MariaDB-log |
+----------+----------------------+

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

I've migrated all s8 eqiad replicas

Change #1202197 had a related patch set uploaded (by Federico Ceratto; author: Federico Ceratto):

[operations/puppet@production] db2161: Migration to MariaDB 10.11

https://gerrit.wikimedia.org/r/1202197

Completed depool of db2161 - Upgrading db2161.codfw.wmnet - fceratto@cumin1003

Change #1202197 merged by Federico Ceratto:

[operations/puppet@production] db2161: Migration to MariaDB 10.11

https://gerrit.wikimedia.org/r/1202197

Start pool of db2161 gradually with 4 steps - Migration of db2161.codfw.wmnet completed - fceratto@cumin1003

Completed pool of db2161 gradually with 4 steps - Migration of db2161.codfw.wmnet completed - fceratto@cumin1003

Migration of db2161.codfw.wmnet completed

Change #1202703 had a related patch set uploaded (by Federico Ceratto; author: Federico Ceratto):

[operations/puppet@production] db2152: Migration to MariaDB 10.11

https://gerrit.wikimedia.org/r/1202703

Completed depool of db2152 - Upgrading db2152.codfw.wmnet - fceratto@cumin1003

Change #1202703 merged by Federico Ceratto:

[operations/puppet@production] db2152: Migration to MariaDB 10.11

https://gerrit.wikimedia.org/r/1202703

Start pool of db2152 gradually with 4 steps - Migration of db2152.codfw.wmnet completed - fceratto@cumin1003

Completed pool of db2152 gradually with 4 steps - Migration of db2152.codfw.wmnet completed - fceratto@cumin1003

Migration of db2152.codfw.wmnet completed

Change #1202724 had a related patch set uploaded (by Federico Ceratto; author: Federico Ceratto):

[operations/puppet@production] db2163: Migration to MariaDB 10.11

https://gerrit.wikimedia.org/r/1202724

Completed depool of db2163 - Upgrading db2163.codfw.wmnet - fceratto@cumin1003

Change #1202724 merged by Federico Ceratto:

[operations/puppet@production] db2163: Migration to MariaDB 10.11

https://gerrit.wikimedia.org/r/1202724

Start pool of db2163 gradually with 4 steps - Migration of db2163.codfw.wmnet completed - fceratto@cumin1003

Completed pool of db2163 gradually with 4 steps - Migration of db2163.codfw.wmnet completed - fceratto@cumin1003

Migration of db2163.codfw.wmnet completed

Change #1202752 had a related patch set uploaded (by Federico Ceratto; author: Federico Ceratto):

[operations/puppet@production] db2154: Migration to MariaDB 10.11

https://gerrit.wikimedia.org/r/1202752

Completed depool of db2154 - Upgrading db2154.codfw.wmnet - fceratto@cumin1003

Change #1202752 merged by Federico Ceratto:

[operations/puppet@production] db2154: Migration to MariaDB 10.11

https://gerrit.wikimedia.org/r/1202752

Start pool of db2154 gradually with 4 steps - Migration of db2154.codfw.wmnet completed - fceratto@cumin1003

Completed pool of db2154 gradually with 4 steps - Migration of db2154.codfw.wmnet completed - fceratto@cumin1003

Migration of db2154.codfw.wmnet completed

Change #1202773 had a related patch set uploaded (by Federico Ceratto; author: Federico Ceratto):

[operations/puppet@production] db2152: Migration to MariaDB 10.11

https://gerrit.wikimedia.org/r/1202773

Change #1202774 had a related patch set uploaded (by Federico Ceratto; author: Federico Ceratto):

[operations/puppet@production] db2164: Migration to MariaDB 10.11

https://gerrit.wikimedia.org/r/1202774

Change #1202775 had a related patch set uploaded (by Federico Ceratto; author: Federico Ceratto):

[operations/puppet@production] db2166: Migration to MariaDB 10.11

https://gerrit.wikimedia.org/r/1202775

Change #1202776 had a related patch set uploaded (by Federico Ceratto; author: Federico Ceratto):

[operations/puppet@production] db2167: Migration to MariaDB 10.11

https://gerrit.wikimedia.org/r/1202776

Change #1202777 had a related patch set uploaded (by Federico Ceratto; author: Federico Ceratto):

[operations/puppet@production] db2181: Migration to MariaDB 10.11

https://gerrit.wikimedia.org/r/1202777

Change #1202773 abandoned by Federico Ceratto:

[operations/puppet@production] db2152: Migration to MariaDB 10.11

https://gerrit.wikimedia.org/r/1202773

Change #1202774 merged by Federico Ceratto:

[operations/puppet@production] db2164: Migration to MariaDB 10.11

https://gerrit.wikimedia.org/r/1202774

Completed depool of db2164 - Upgrading db2164.codfw.wmnet - fceratto@cumin1003

Start pool of db2164 gradually with 4 steps - Migration of db2164.codfw.wmnet completed - fceratto@cumin1003

Completed pool of db2164 gradually with 4 steps - Migration of db2164.codfw.wmnet completed - fceratto@cumin1003

Migration of db2164.codfw.wmnet completed

Completed depool of db2166 - Upgrading db2166.codfw.wmnet - fceratto@cumin1003

Change #1202775 merged by Federico Ceratto:

[operations/puppet@production] db2166: Migration to MariaDB 10.11

https://gerrit.wikimedia.org/r/1202775

Start pool of db2166 gradually with 4 steps - Migration of db2166.codfw.wmnet completed - fceratto@cumin1003

Completed pool of db2166 gradually with 4 steps - Migration of db2166.codfw.wmnet completed - fceratto@cumin1003

Migration of db2166.codfw.wmnet completed

Completed depool of db2167 - Upgrading db2167.codfw.wmnet - fceratto@cumin1003

Change #1202776 merged by Federico Ceratto:

[operations/puppet@production] db2167: Migration to MariaDB 10.11

https://gerrit.wikimedia.org/r/1202776

Start pool of db2167 gradually with 4 steps - Migration of db2167.codfw.wmnet completed - fceratto@cumin1003

Completed pool of db2167 gradually with 4 steps - Migration of db2167.codfw.wmnet completed - fceratto@cumin1003

Migration of db2167.codfw.wmnet completed

Completed depool of db2181 - Upgrading db2181.codfw.wmnet - fceratto@cumin1003

Change #1202777 merged by Federico Ceratto:

[operations/puppet@production] db2181: Migration to MariaDB 10.11

https://gerrit.wikimedia.org/r/1202777

Start pool of db2181 gradually with 4 steps - Migration of db2181.codfw.wmnet completed - fceratto@cumin1003

Completed pool of db2181 gradually with 4 steps - Migration of db2181.codfw.wmnet completed - fceratto@cumin1003

Migration of db2181.codfw.wmnet completed

The process is basically:

  • Depool and downtime
  • Create the puppet patch
  • Disable puppet
  • stop slave;
  • stop mariadb service
  • sudo apt remove wmf-mariadb106
  • merge puppet patch and run puppet-merge on puppetserver
  • enable puppet
  • run-puppet-agent
  • start mariadb service
  • mysql_upgrade (this is really important)
  • start slave;
  • once caught up, repool

As part of a proper major version upgrade, I would add a cleanup restart after mysql_upgrade. It makes sure logs come up clean and it makes sure we don't have any logging issue like we had in the past, new tables get reloaded satisfactorily.

Completed depool of db2165 - Upgrading db2165.codfw.wmnet - fceratto@cumin1003

Change #1204746 had a related patch set uploaded (by Federico Ceratto; author: Federico Ceratto):

[operations/puppet@production] db2165: Migration to MariaDB 10.11

https://gerrit.wikimedia.org/r/1204746

Change #1204746 merged by Federico Ceratto:

[operations/puppet@production] db2165: Migration to MariaDB 10.11

https://gerrit.wikimedia.org/r/1204746

Start pool of db2165 gradually with 4 steps - Migration of db2165.codfw.wmnet completed - fceratto@cumin1003

Completed pool of db2165 gradually with 4 steps - Migration of db2165.codfw.wmnet completed - fceratto@cumin1003

Migration of db2165.codfw.wmnet completed