install krb2002
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	RobH
	Apr 5 2022, 4:18 PM

Description

This task will track the racking, setup, and OS installation of krb2002

Hostname / Racking / Installation Details

Hostnames: krb2002
Racking Proposal: Any 1G rack, there are no restrictions where to place them
Networking Setup:* 1G in private VLAN, with AAAA records
Partitioning/Raid: raid1-2dev.cfg
OS Distro:** Bullseye

Per host setup checklist

Each host should have its own setup checklist copied and pasted into the list below.

krb2002: D5/U13 ge-5/0/12

- receive in system on procurement task T303411 & in coupa
- rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
- add mgmt dns (asset tag and hostname) and production dns entries in netbox, run cookbook sre.dns.netbox.
- network port setup via netbox, run homer from an active cumin host to commit
- bios/drac/serial setup/testing, see Lifecycle Steps & Automatic BIOS setup details
- firmware update (idrac, bios, network, raid controller)
- operations/puppet update - this should include updates to netboot.pp, and site.pp role(insetup) or cp systems use role(insetup::nofirm).
- OS installation & initital puppet run via sre.hosts.reimage cookbook.

Details

	Subject	Repo	Branch	Lines +/-
	Add new pki and krb node to site.pp and netboot.cfg	operations/puppet	production	+11 -0

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
		Unknown Object (Task)
Resolved	MoritzMuehlenhoff	T331695 Migrate the KDCs to Bullseye
Resolved	Papaul	T305488 Q4:(Need By: TBD) rack/setup/install krb2002

Event Timeline

RobH created this task.Apr 5 2022, 4:18 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 5 2022, 4:18 PM

RobH added a parent task: Unknown Object (Task).Apr 5 2022, 4:18 PM

RobH mentioned this in Unknown Object (Task).

RobH moved this task from Backlog to Racking Tasks on the ops-codfw board.

Maintenance_bot added a project: SRE.Apr 5 2022, 4:29 PM

Papaul updated the task description. (Show Details)May 2 2022, 6:13 PM

RobH unsubscribed.May 2 2022, 6:14 PM

Change 788433 had a related patch set uploaded (by Papaul; author: Papaul):

[operations/puppet@production] Add nes pki and krb node to site.pp and netboot.cfg

https://gerrit.wikimedia.org/r/788433

gerritbot added a project: Patch-For-Review.May 2 2022, 10:14 PM

Change 788433 merged by Papaul:

[operations/puppet@production] Add new pki and krb node to site.pp and netboot.cfg

https://gerrit.wikimedia.org/r/788433

Maintenance_bot removed a project: Patch-For-Review.May 2 2022, 10:31 PM

Papaul updated the task description. (Show Details)May 2 2022, 10:39 PM

Cookbook cookbooks.sre.hosts.reimage was started by pt1979@cumin2002 for host krb2002.codfw.wmnet with OS bullseye

Cookbook cookbooks.sre.hosts.reimage started by pt1979@cumin2002 for host krb2002.codfw.wmnet with OS bullseye completed:

krb2002 (PASS)
- Removed from Puppet and PuppetDB if present
- Deleted any existing Puppet certificate
- Removed from Debmonitor if present
- Forced PXE for next reboot
- Host rebooted via IPMI
- Host up (Debian installer)
- Host up (new fresh bullseye OS)
- Generated Puppet certificate
- Signed new Puppet certificate
- Run Puppet in NOOP mode to populate exported resources in PuppetDB
- Found Nagios_host resource for this host in PuppetDB
- Downtimed the new host on Icinga/Alertmanager
- First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202205022240_pt1979_1444634_krb2002.out
- Checked BIOS boot parameters are back to normal
- configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
- Rebooted
- Automatic Puppet run was successful
- Forced a re-check of all Icinga services for the host
- Icinga status is optimal
- Icinga downtime removed
- Updated Netbox data from PuppetDB
- Updated Netbox status planned -> staged

@MoritzMuehlenhoff this is complete

Cookbook cookbooks.sre.hosts.reimage was started by jmm@cumin2002 for host krb2002.codfw.wmnet with OS bullseye

Cookbook cookbooks.sre.hosts.reimage started by jmm@cumin2002 for host krb2002.codfw.wmnet with OS bullseye completed:

krb2002 (PASS)
- Downtimed on Icinga/Alertmanager
- Disabled Puppet
- Removed from Puppet and PuppetDB if present
- Deleted any existing Puppet certificate
- Removed from Debmonitor if present
- Forced PXE for next reboot
- Host rebooted via IPMI
- Host up (Debian installer)
- Host up (new fresh bullseye OS)
- Generated Puppet certificate
- Signed new Puppet certificate
- Run Puppet in NOOP mode to populate exported resources in PuppetDB
- Found Nagios_host resource for this host in PuppetDB
- Downtimed the new host on Icinga/Alertmanager
- Removed previous downtime on Alertmanager (old OS)
- First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202302010949_jmm_2996961_krb2002.out
- Checked BIOS boot parameters are back to normal
- configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
- Rebooted
- Automatic Puppet run was successful
- Forced a re-check of all Icinga services for the host
- Icinga status is optimal
- Icinga downtime removed
- Updated Netbox data from PuppetDB

MoritzMuehlenhoff added a parent task: T331695: Migrate the KDCs to Bullseye.Mar 10 2023, 8:11 AM

Q4:(Need By: TBD) rack/setup/install krb2002Closed, ResolvedPublicActions