PuppetZeroResources (planet1003 new bullseye host)
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	phaultfinder
	Nov 23 2023, 2:37 AM

Description

Common information

dashboard: https://grafana.wikimedia.org/d/yOxVDGvWk/puppet
description: Zero Puppet resources on planet1003:9100
runbook: https://puppetboard.wikimedia.org/nodes?status=failed
summary: Puppet has failed generate resources on planet1003:9100

alertname: PuppetZeroResources
cluster: misc
instance: planet1003:9100
job: node
prometheus: ops
severity: critical
site: eqiad
source: prometheus
team: serviceops-collab

Firing alerts

dashboard: https://grafana.wikimedia.org/d/yOxVDGvWk/puppet
description: Zero Puppet resources on planet1003:9100
runbook: https://puppetboard.wikimedia.org/nodes?status=failed
summary: Puppet has failed generate resources on planet1003:9100
alertname: PuppetZeroResources
cluster: misc
instance: planet1003:9100
job: node
prometheus: ops
severity: critical
site: eqiad
source: prometheus
team: serviceops-collab
Source

Related Objects

Mentioned Here: T348392: Migrate planet servers to bullseye or bookworm

Event Timeline

phaultfinder created this task.Nov 23 2023, 2:37 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 23 2023, 2:38 AM

I think @Dzahn added the new host plane1003 for the bullseye migration (T348392).
Puppet still has some certificate errors on the host. A speculative guess: is certificate signing different on puppet7? SAL states:

2023-11-22 22:34 <mutante> puppetserver1001 - manually signed puppet cert request for planet1003

Jelto renamed this task from PuppetZeroResources to PuppetZeroResources (planet1003 new bullseye host).Nov 23 2023, 9:30 AM

Jelto moved this task from Incoming to Work in Progress on the collaboration-services board.

Dzahn changed the task status from Open to In Progress.Nov 27 2023, 6:07 PM

wmf-reimage cookbook worked this time unlike last week. cert was revoked and new cert signed by cookbook.

puppet working now.

PuppetZeroResources (planet1003 new bullseye host)Closed, ResolvedPublicActions