PuppetFailure - Contint
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	phaultfinder
	Oct 31 2023, 8:04 AM

Description

Common information

dashboard: https://grafana.wikimedia.org/d/yOxVDGvWk/puppet
runbook: https://puppetboard.wikimedia.org/nodes?status=failed

alertname: PuppetFailure
cluster: ci
job: node
prometheus: ops
severity: critical
source: prometheus
team: serviceops-collab

Firing alerts

dashboard: https://grafana.wikimedia.org/d/yOxVDGvWk/puppet
description: Puppet failure on contint2002:9100
runbook: https://puppetboard.wikimedia.org/nodes?status=failed
summary: Puppet has failed on contint2002:9100
alertname: PuppetFailure
cluster: ci
instance: contint2002:9100
job: node
prometheus: ops
severity: critical
site: codfw
source: prometheus
team: serviceops-collab
Source

dashboard: https://grafana.wikimedia.org/d/yOxVDGvWk/puppet
description: Puppet failure on contint1002:9100
runbook: https://puppetboard.wikimedia.org/nodes?status=failed
summary: Puppet has failed on contint1002:9100
alertname: PuppetFailure
cluster: ci
instance: contint1002:9100
job: node
prometheus: ops
severity: critical
site: eqiad
source: prometheus
team: serviceops-collab
Source

Related Objects

Mentioned Here: T350118: Investigate PKI errors

Event Timeline

phaultfinder created this task.Oct 31 2023, 8:04 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 31 2023, 8:05 AM

phaultfinder updated the task description. (Show Details)Oct 31 2023, 8:29 AM

LSobanski renamed this task from PuppetFailure to PuppetFailure - Contint.Oct 31 2023, 10:06 AM

I did not get to see the actual error but it seems to be gone, based on the puppet board, which doesn't find contint hosts with failed puppet an longer.

https://puppetboard.wikimedia.org/nodes?status=failed

Most likely related to T350118: Investigate PKI errors

LSobanski closed this task as Resolved.Nov 1 2023, 12:53 PM

LSobanski claimed this task.

PuppetFailure - ContintClosed, ResolvedPublicActions