PKI system is unable to serve new certificates to debmonitor / other systems, causing puppet failures across the fleet.
Closed, DuplicatePublic
Actions

Assigned To

None

Authored By

	Joe
	Oct 31 2023, 8:19 AM

Description

Since this change was merged and applied

It seems that the PKI subsystem has been unable to work:

ocsp refresh jobs fail to update the database as it seems the server doesn't have the old puppet CA in the main certstore anymore
Other servers don't seem to be able to refresh their certs from the PKI

As a consequence, puppet runs are broken across the fleet since last night at 17:40 UTC.

	Subject	Repo	Branch	Lines +/-
	P:pki: use wmf-ca-certificates	operations/puppet	production	+2 -2

Joe triaged this task as Unbreak Now! priority.Oct 31 2023, 8:20 AM

Joe updated the task description. (Show Details)

Change 970267 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] P:pki: use wmf-ca-certificates

Change 970267 merged by Jbond:

[operations/puppet@production] P:pki: use wmf-ca-certificates

The immediate incident has been resolved ill complete the investigation in T350118