Page MenuHomePhabricator

codfw: request for a decom'ed R440 - Config C
Closed, ResolvedPublic

Description

We would like to request a machine in codfw that is decom'ed but not deracked yet.

type: R440 - Config C (if available)

it should be named contint2003.wikimedia.org

resembling existing contint2002.wikimedia.org (https://netbox.wikimedia.org/search/?q=contint2002)

This would unblock us at T418109 and would be temporary. Thank you very much!

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Change #1244743 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/puppet@production] site: add contint1003/2003 with insetup collab role

https://gerrit.wikimedia.org/r/1244743

servers racked and the idrac ip is set up. giving me a bit of trouble with the provisioning script but if you wanna steal it while i'm at lunch, i will not argue.

Dzahn removed Dzahn as the assignee of this task.Mar 2 2026, 6:48 PM

Change #1244743 merged by Dzahn:

[operations/puppet@production] site: add contint1003/2003 with insetup collab role

https://gerrit.wikimedia.org/r/1244743

Cookbook cookbooks.sre.hosts.reimage was started by jhancock@cumin2002 for host contint2003.wikimedia.org with OS bookworm

Cookbook cookbooks.sre.hosts.reimage started by jhancock@cumin2002 for host contint2003.wikimedia.org with OS bookworm executed with errors:

  • contint2003 (FAIL)
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced UEFI HTTP Boot for next reboot
    • Host rebooted via Redfish
    • Host up (Debian installer)
    • The reimage failed, see the cookbook logs for the details. You can also try typing "sudo install-console contint2003.wikimedia.org" to get a root shell, but depending on the failure this may not work.

Cookbook cookbooks.sre.hosts.reimage was started by jhancock@cumin2002 for host contint2003.wikimedia.org with OS bookworm

Cookbook cookbooks.sre.hosts.reimage started by jhancock@cumin2002 for host contint2003.wikimedia.org with OS bookworm completed:

  • contint2003 (PASS)
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Checked BIOS boot parameters are back to normal
    • Host up (new fresh bookworm OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202603051659_jhancock_3916585_contint2003.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
    • Updated Netbox status planned -> active
    • The sre.puppet.sync-netbox-hiera cookbook was run successfully

@Jhancock.wm Thank you very much! Taking over:)

Dzahn claimed this task.

Can SSH to the machine. wfm:) Further puppet setup will be over here: T418521