- pc2017
- pc2018
- pc1017
- pc1018
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Marostegui | T422365 Migration to Debian Trixie of production database-related hosts | |||
| Resolved | Marostegui | T422368 Migrate parsercache to Debian Trixie | |||
| Resolved | Marostegui | T429178 Upgrade pc7 and pc8 to Debian Trixie |
Event Timeline
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2026-06-22T08:12:53Z] <marostegui@cumin1003> dbmaint on pc7@codfw T429178
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2026-06-22T08:17:16Z] <marostegui@cumin1003> dbmaint on pc7@codfw T429178
Comment Actions
Cookbook cookbooks.sre.hosts.reimage was started by marostegui@cumin1003 for host pc2017.codfw.wmnet with OS trixie
Comment Actions
Cookbook cookbooks.sre.hosts.reimage started by marostegui@cumin1003 for host pc2017.codfw.wmnet with OS trixie completed:
- pc2017 (WARN)
- Downtimed on Icinga/Alertmanager
- Disabled Puppet
- Removed from Puppet and PuppetDB if present and deleted any certificates
- Removed from Debmonitor if present
- Forced PXE for next reboot
- Host rebooted via IPMI
- Host up (Debian installer)
- Checked BIOS boot parameters are back to normal
- Host up (new fresh trixie OS)
- Generated Puppet certificate
- Signed new Puppet certificate
- Run Puppet in NOOP mode to populate exported resources in PuppetDB
- Found Nagios_host resource for this host in PuppetDB
- Downtimed the new host on Icinga/Alertmanager
- Removed previous downtime on Alertmanager (old OS)
- First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606220844_marostegui_1745688_pc2017.out
- configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
- Rebooted
- Automatic Puppet run was successful
- Forced a re-check of all Icinga services for the host
- Icinga status is not optimal, downtime not removed
- Updated Netbox data from PuppetDB
Comment Actions
Cookbook cookbooks.sre.hosts.reimage was started by marostegui@cumin1003 for host pc1017.eqiad.wmnet with OS trixie
Comment Actions
Cookbook cookbooks.sre.hosts.reimage started by marostegui@cumin1003 for host pc1017.eqiad.wmnet with OS trixie completed:
- pc1017 (WARN)
- Downtimed on Icinga/Alertmanager
- Disabled Puppet
- Removed from Puppet and PuppetDB if present and deleted any certificates
- Removed from Debmonitor if present
- Forced PXE for next reboot
- Host rebooted via IPMI
- Host up (Debian installer)
- Checked BIOS boot parameters are back to normal
- Host up (new fresh trixie OS)
- Generated Puppet certificate
- Signed new Puppet certificate
- Run Puppet in NOOP mode to populate exported resources in PuppetDB
- Found Nagios_host resource for this host in PuppetDB
- Downtimed the new host on Icinga/Alertmanager
- Removed previous downtime on Alertmanager (old OS)
- First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606220935_marostegui_1783156_pc1017.out
- configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
- Rebooted
- Automatic Puppet run was successful
- Forced a re-check of all Icinga services for the host
- Icinga status is not optimal, downtime not removed
- Updated Netbox data from PuppetDB
Comment Actions
Cookbook cookbooks.sre.hosts.reimage was started by marostegui@cumin1003 for host pc2018.codfw.wmnet with OS trixie
Comment Actions
Cookbook cookbooks.sre.hosts.reimage started by marostegui@cumin1003 for host pc2018.codfw.wmnet with OS trixie completed:
- pc2018 (WARN)
- Downtimed on Icinga/Alertmanager
- Disabled Puppet
- Removed from Puppet and PuppetDB if present and deleted any certificates
- Removed from Debmonitor if present
- Forced PXE for next reboot
- Host rebooted via IPMI
- Host up (Debian installer)
- Checked BIOS boot parameters are back to normal
- Host up (new fresh trixie OS)
- Generated Puppet certificate
- Signed new Puppet certificate
- Run Puppet in NOOP mode to populate exported resources in PuppetDB
- Found Nagios_host resource for this host in PuppetDB
- Downtimed the new host on Icinga/Alertmanager
- Removed previous downtime on Alertmanager (old OS)
- First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606230604_marostegui_2202888_pc2018.out
- configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
- Rebooted
- Automatic Puppet run was successful
- Forced a re-check of all Icinga services for the host
- Icinga status is not optimal, downtime not removed
- Updated Netbox data from PuppetDB
Comment Actions
Cookbook cookbooks.sre.hosts.reimage was started by marostegui@cumin1003 for host pc1018.eqiad.wmnet with OS trixie
Comment Actions
Cookbook cookbooks.sre.hosts.reimage started by marostegui@cumin1003 for host pc1018.eqiad.wmnet with OS trixie completed:
- pc1018 (WARN)
- Downtimed on Icinga/Alertmanager
- Disabled Puppet
- Removed from Puppet and PuppetDB if present and deleted any certificates
- Removed from Debmonitor if present
- Forced PXE for next reboot
- Host rebooted via IPMI
- Host up (Debian installer)
- Checked BIOS boot parameters are back to normal
- Host up (new fresh trixie OS)
- Generated Puppet certificate
- Signed new Puppet certificate
- Run Puppet in NOOP mode to populate exported resources in PuppetDB
- Found Nagios_host resource for this host in PuppetDB
- Downtimed the new host on Icinga/Alertmanager
- Removed previous downtime on Alertmanager (old OS)
- First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606230700_marostegui_2232444_pc1018.out
- configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
- Rebooted
- Automatic Puppet run was successful
- Forced a re-check of all Icinga services for the host
- Icinga status is not optimal, downtime not removed
- Updated Netbox data from PuppetDB