Page MenuHomePhabricator
Create Task
Maniphest T358746

Upgrade es5 to MariaDB 10.6
Closed, ResolvedPublic
Actions

Description

  • es2023
  • es2024
  • es2025
  • es1023
  • es1024
  • es1025

Details

Related Changes in Gerrit:
SubjectAuthorRepoBranchLines +/-
es1024: Migrate to 10.6Marosteguioperations/puppetproduction+1 -0
es2023: Migrate to MariaDB 10.6Marosteguioperations/puppetproduction+1 -0
es2024: Migrate to MariaDB 10.6Marosteguioperations/puppetproduction+1 -0
es2025: Migrate to MariaDB 10.6Marosteguioperations/puppetproduction+1 -0
es1025: Migrate to MariaDB 10.6Marosteguioperations/puppetproduction+1 -1
Customize query in gerrit

Related Objects
Search...

Event Timeline

Marostegui created this task.Feb 29 2024, 10:01 AM
Marostegui triaged this task as Medium priority.
Marostegui moved this task from Triage to In progress on the DBA board.
Marostegui mentioned this in T356960: Upgrade hosts to MariaDB 10.6.
gerritbot added a comment.Mar 6 2024, 6:22 AM

Change 1009152 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/puppet@production] es1025: Migrate to MariaDB 10.6

https://gerrit.wikimedia.org/r/1009152

gerritbot added a project: Patch-For-Review.Mar 6 2024, 6:22 AM
gerritbot added a comment.Mar 6 2024, 6:24 AM

Change 1009152 merged by Marostegui:

[operations/puppet@production] es1025: Migrate to MariaDB 10.6

https://gerrit.wikimedia.org/r/1009152

Marostegui updated the task description. (Show Details)Mar 6 2024, 6:27 AM
Maintenance_bot removed a project: Patch-For-Review.Mar 6 2024, 6:30 AM
gerritbot added a comment.Mar 20 2024, 4:56 PM

Change 1013103 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/puppet@production] es2025: Migrate to MariaDB 10.6

https://gerrit.wikimedia.org/r/1013103

gerritbot added a project: Patch-For-Review.Mar 20 2024, 4:56 PM
Stashbot added a comment.Mar 20 2024, 4:57 PM

Mentioned in SAL (#wikimedia-operations) [2024-03-20T16:57:11Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es2025 T358746', diff saved to https://phabricator.wikimedia.org/P58826 and previous config saved to /var/cache/conftool/dbconfig/20240320-165710-root.json

gerritbot added a comment.Mar 20 2024, 4:58 PM

Change 1013103 merged by Marostegui:

[operations/puppet@production] es2025: Migrate to MariaDB 10.6

https://gerrit.wikimedia.org/r/1013103

Stashbot added a comment.Mar 20 2024, 5:03 PM

Mentioned in SAL (#wikimedia-operations) [2024-03-20T17:03:33Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repool es2025 T358746', diff saved to https://phabricator.wikimedia.org/P58827 and previous config saved to /var/cache/conftool/dbconfig/20240320-170332-root.json

Marostegui updated the task description. (Show Details)Mar 20 2024, 5:03 PM
Stashbot added a comment.Mar 20 2024, 5:04 PM

Mentioned in SAL (#wikimedia-operations) [2024-03-20T17:04:13Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es2024 T358746', diff saved to https://phabricator.wikimedia.org/P58828 and previous config saved to /var/cache/conftool/dbconfig/20240320-170413-root.json

gerritbot added a comment.Mar 20 2024, 5:05 PM

Change 1013106 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/puppet@production] es2024: Migrate to MariaDB 10.6

https://gerrit.wikimedia.org/r/1013106

gerritbot added a comment.Mar 20 2024, 5:07 PM

Change 1013106 merged by Marostegui:

[operations/puppet@production] es2024: Migrate to MariaDB 10.6

https://gerrit.wikimedia.org/r/1013106

Marostegui updated the task description. (Show Details)Mar 20 2024, 5:08 PM
Stashbot added a comment.Mar 20 2024, 5:13 PM

Mentioned in SAL (#wikimedia-operations) [2024-03-20T17:13:57Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repool es2024 T358746', diff saved to https://phabricator.wikimedia.org/P58829 and previous config saved to /var/cache/conftool/dbconfig/20240320-171356-root.json

Maintenance_bot removed a project: Patch-For-Review.Mar 20 2024, 5:31 PM
gerritbot added a comment.Mar 21 2024, 6:20 AM

Change 1013162 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/puppet@production] es2023: Migrate to MariaDB 10.6

https://gerrit.wikimedia.org/r/1013162

gerritbot added a project: Patch-For-Review.Mar 21 2024, 6:20 AM

Change 1013162 merged by Marostegui:

[operations/puppet@production] es2023: Migrate to MariaDB 10.6

https://gerrit.wikimedia.org/r/1013162

Marostegui updated the task description. (Show Details)Mar 21 2024, 6:24 AM
Maintenance_bot removed a project: Patch-For-Review.Mar 21 2024, 6:30 AM
gerritbot added a comment.Apr 2 2024, 4:55 AM

Change #1016076 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/puppet@production] es1024: Migrate to 10.6

https://gerrit.wikimedia.org/r/1016076

gerritbot added a project: Patch-For-Review.Apr 2 2024, 4:55 AM
Stashbot added a comment.Apr 2 2024, 4:56 AM

Mentioned in SAL (#wikimedia-operations) [2024-04-02T04:56:00Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es1024 T358746', diff saved to https://phabricator.wikimedia.org/P59108 and previous config saved to /var/cache/conftool/dbconfig/20240402-045559-root.json

gerritbot added a comment.Apr 2 2024, 4:57 AM

Change #1016076 merged by Marostegui:

[operations/puppet@production] es1024: Migrate to 10.6

https://gerrit.wikimedia.org/r/1016076

Marostegui updated the task description. (Show Details)Apr 2 2024, 5:06 AM
Marostegui closed subtask T361548: Switchover es5 master (es1023 -> es1024) as Resolved.Apr 29 2024, 6:00 AM
ops-monitoring-bot added a comment.Apr 29 2024, 6:02 AM

Cookbook cookbooks.sre.hosts.reimage was started by marostegui@cumin1002 for host es1023.eqiad.wmnet with OS bookworm

Maintenance_bot removed a project: Patch-For-Review.Apr 29 2024, 6:31 AM
Marostegui updated the task description. (Show Details)Apr 29 2024, 6:35 AM
ops-monitoring-bot added a comment.Apr 29 2024, 6:43 AM

Cookbook cookbooks.sre.hosts.reimage started by marostegui@cumin1002 for host es1023.eqiad.wmnet with OS bookworm completed:

  • es1023 (WARN)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Add puppet_version metadata to Debian installer
    • Checked BIOS boot parameters are back to normal
    • Host up (new fresh bookworm OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202404290621_marostegui_2976652_es1023.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is not optimal, downtime not removed
    • Updated Netbox data from PuppetDB
Marostegui closed this task as Resolved.Apr 29 2024, 6:45 AM

This is done

Maintenance_bot moved this task from In progress to Done on the DBA board.Apr 29 2024, 7:29 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits