Page MenuHomePhabricator

Upgrade analytics dbstore databases to Buster and Mariadb 10.4
Closed, ResolvedPublic

Description

I would like to upgrade dbstore1003, dbstore1004 and dbstore1005 to Buster and MariaDB 10.4
We already have lots of hosts in production running this setup (including multi-instance hosts like dbstore100X).

I would do one at the time and will leave a few days between upgrading them.
Normally, upgrading a host takes around 15-30 minutes, so MySQL won't be accessible during this time.
@Nuria @elukey any objection from your side? Any preference from which host I should start from?

  • dbstore1003 (s1, s5, s7)
  • dbstore1004 (s2, s3, s4)
  • dbstore1005 (s6, s8, x1, staging database)

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 9 2020, 12:34 PM
Marostegui triaged this task as Medium priority.Jun 9 2020, 12:34 PM
Marostegui moved this task from Triage to Next on the DBA board.

No preference! I think that given the limited downtime it can be done anytime, no problems from my side. Do you require help in doing anything?

No preference! I think that given the limited downtime it can be done anytime, no problems from my side. Do you require help in doing anything?

Thank you! I will start tomorrow then with dbstore1003. No help is needed, thank you! :)

Change 604551 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] mariadb: Reimage dbstore1003 with Buster

https://gerrit.wikimedia.org/r/604551

Change 604551 merged by Marostegui:
[operations/puppet@production] mariadb: Reimage dbstore1003 with Buster

https://gerrit.wikimedia.org/r/604551

Mentioned in SAL (#wikimedia-operations) [2020-06-11T07:07:52Z] <marostegui> Stop MySQL on dbstore1003 for reimage - T254870

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['dbstore1003.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202006110712_marostegui_199022.log.

Completed auto-reimage of hosts:

['dbstore1003.eqiad.wmnet']

and were ALL successful.

Marostegui updated the task description. (Show Details)Jun 11 2020, 7:44 AM

Change 605731 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] mariadb: Reimage dbstore1004 to Buster

https://gerrit.wikimedia.org/r/605731

Change 605731 merged by Marostegui:
[operations/puppet@production] mariadb: Reimage dbstore1004 to Buster

https://gerrit.wikimedia.org/r/605731

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['dbstore1004.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202006160449_marostegui_211436.log.

Completed auto-reimage of hosts:

['dbstore1004.eqiad.wmnet']

and were ALL successful.

Marostegui updated the task description. (Show Details)Jun 16 2020, 5:12 AM

Change 605809 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] dbstore1004: Enable notifications

https://gerrit.wikimedia.org/r/605809

Change 605809 merged by Marostegui:
[operations/puppet@production] dbstore1004: Enable notifications

https://gerrit.wikimedia.org/r/605809

Change 606851 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] dbstore1005: Disable notifications

https://gerrit.wikimedia.org/r/606851

Change 606851 merged by Marostegui:
[operations/puppet@production] install_server: Remiage dbstore1005 to Buster

https://gerrit.wikimedia.org/r/606851

Mentioned in SAL (#wikimedia-operations) [2020-06-22T06:06:09Z] <marostegui> Stop MySQL on dbstore1005 for reimage to Buster - T254870

Script wmf-auto-reimage was launched by marostegui on cumin2001.codfw.wmnet for hosts:

['dbstore1005.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202006220611_marostegui_27761.log.

Completed auto-reimage of hosts:

['dbstore1005.eqiad.wmnet']

and were ALL successful.

Change 606853 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] dbstore1005: Enable notifications

https://gerrit.wikimedia.org/r/606853

Change 606853 merged by Marostegui:
[operations/puppet@production] dbstore1005: Enable notifications

https://gerrit.wikimedia.org/r/606853

Marostegui closed this task as Resolved.Jun 22 2020, 7:00 AM
Marostegui claimed this task.
Marostegui updated the task description. (Show Details)

All done!