We've tested this role on pc1, seems to be working fine, I don't expect issues on msX but let's test on ms1
- db2142
- db1152
Description
Description
Details
Details
Related Changes in Gerrit:
| Subject | Author | Repo | Branch | Lines +/- | |
|---|---|---|---|---|---|
| db1152: Disable notifications | Marostegui | operations/puppet | production | +1 -0 | |
| db2142: Disable notifications | Marostegui | operations/puppet | production | +1 -0 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Marostegui | T422365 Migration to Debian Trixie of production database-related hosts | |||
| Resolved | Marostegui | T423874 Migrate msX to Debian Trixie | |||
| Resolved | Marostegui | T418561 Test Debian Trixie on ms1 |
Event Timeline
Comment Actions
Depooled db1152.eqiad.wmnet and db2142.codfw.wmnet Upgrade - marostegui@cumin1003 - T418561
Comment Actions
Change #1268200 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/puppet@production] db2142: Disable notifications
Comment Actions
Change #1268200 merged by Marostegui:
[operations/puppet@production] db2142: Disable notifications
Comment Actions
Cookbook cookbooks.sre.hosts.reimage was started by marostegui@cumin1003 for host db2142.codfw.wmnet with OS trixie
Comment Actions
Cookbook cookbooks.sre.hosts.reimage started by marostegui@cumin1003 for host db2142.codfw.wmnet with OS trixie completed:
- db2142 (PASS)
- Downtimed on Icinga/Alertmanager
- Disabled Puppet
- Removed from Puppet and PuppetDB if present and deleted any certificates
- Removed from Debmonitor if present
- Forced PXE for next reboot
- Host rebooted via IPMI
- Host up (Debian installer)
- Checked BIOS boot parameters are back to normal
- Host up (new fresh trixie OS)
- Generated Puppet certificate
- Signed new Puppet certificate
- Run Puppet in NOOP mode to populate exported resources in PuppetDB
- Found Nagios_host resource for this host in PuppetDB
- Downtimed the new host on Icinga/Alertmanager
- Removed previous downtime on Alertmanager (old OS)
- First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202604060954_marostegui_3088619_db2142.out
- configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
- Rebooted
- Automatic Puppet run was successful
- Forced a re-check of all Icinga services for the host
- Icinga status is optimal
- Icinga downtime removed
- Updated Netbox data from PuppetDB
Comment Actions
Depooled db1152.eqiad.wmnet and db2142.codfw.wmnet After upgrade db2142 - marostegui@cumin1003 - T418561
Comment Actions
Change #1268708 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/puppet@production] db1152: Disable notifications
Comment Actions
Change #1268708 merged by Marostegui:
[operations/puppet@production] db1152: Disable notifications
Comment Actions
Cookbook cookbooks.sre.hosts.reimage was started by marostegui@cumin1003 for host db1152.eqiad.wmnet with OS trixie
Comment Actions
Cookbook cookbooks.sre.hosts.reimage started by marostegui@cumin1003 for host db1152.eqiad.wmnet with OS trixie completed:
- db1152 (PASS)
- Downtimed on Icinga/Alertmanager
- Disabled Puppet
- Removed from Puppet and PuppetDB if present and deleted any certificates
- Removed from Debmonitor if present
- Forced PXE for next reboot
- Host rebooted via IPMI
- Host up (Debian installer)
- Checked BIOS boot parameters are back to normal
- Host up (new fresh trixie OS)
- Generated Puppet certificate
- Signed new Puppet certificate
- Run Puppet in NOOP mode to populate exported resources in PuppetDB
- Found Nagios_host resource for this host in PuppetDB
- Downtimed the new host on Icinga/Alertmanager
- Removed previous downtime on Alertmanager (old OS)
- First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202604080533_marostegui_3676258_db1152.out
- configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
- Rebooted
- Automatic Puppet run was successful
- Forced a re-check of all Icinga services for the host
- Icinga status is optimal
- Icinga downtime removed
- Updated Netbox data from PuppetDB