Page MenuHomePhabricator

Document all uses of the puppetCA certificate
Closed, ResolvedPublic

Description

As we are due to update the puppet CA we will need to know all locations where the certificate is currently used. The following is a good list of places to start looking but may not be complete

  • keys created using cergen
  • users of base::expose_puppet_certs
  • the few users we have that are referencing either puppet_ssldir() or manually hardcoding the /var/lib/puppet/ssl directory
  • Any helm charts that need our CA cert: eventgate

The following services make reference to Puppet_Internal_CA.crt

Related Objects

StatusSubtypeAssignedTask
Resolvedjbond
Resolvedjbond
ResolvedRLazarus
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedTrizek-WMF
ResolvedMarostegui
ResolvedTrizek-WMF
ResolvedMarostegui
ResolvedTrizek-WMF
ResolvedMarostegui
ResolvedTrizek-WMF
ResolvedMarostegui
ResolvedTrizek-WMF
ResolvedMarostegui
ResolvedMarostegui
ResolvedJAllemandou
ResolvedBstorm
ResolvedTrizek-WMF
ResolvedMarostegui
ResolvedTrizek-WMF
Resolvedjbond

Event Timeline

jbond triaged this task as Medium priority.Nov 4 2019, 2:36 PM
jbond created this task.

base::expose_puppet_certs - This can be ignored as it relates to the client key pairs and not the CA public certificate which is the file which is changing. Further i think keys created using cergen can also be largley ignored. We dont much care for certs that have been generated we just need to ensure that anything validating theses certificates also has the new public CA i.e references to {/usr/local/share/ca-certificates/,/etc/ssl/certs/}Puppet_Internal_CA.crt

Acme-chief nginx config probably?

As far as etcd is concerned, a rolling restart should be enough to ensure the new CA is picked up. I will take care of that.

As far as etcd is concerned, a rolling restart should be enough to ensure the new CA is picked up. I will take care of that.

Thanks, ill ping T237362 when the the CR is merged

Change 548706 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] profile::base: reorder class

https://gerrit.wikimedia.org/r/548706

@Eevans moritz mentioned there maybe some cassandra consideration to take into account and you could enlighten me as to what they are :)

CDanis updated the task description. (Show Details)

@Eevans moritz mentioned there maybe some cassandra consideration to take into account and you could enlighten me as to what they are :)

Cassandra uses its own CA, which is per-installation. It has no relationship with the main CA we use for puppet. One could say we should maybe check the expiration of those CAs as well, but I think it's already covered by monitoring.

Change 548706 merged by Jbond:
[operations/puppet@production] profile::base: reorder class

https://gerrit.wikimedia.org/r/548706

The calico/node service, and the kube-controller-manager service will need to be restarted on the kubernetes workers and masters respectively.

Ottomata updated the task description. (Show Details)
Ottomata updated the task description. (Show Details)

Change 554584 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/deployment-charts@master] eventgate-logging values: update puppet CA

https://gerrit.wikimedia.org/r/554584

Change 554584 merged by Ottomata:
[operations/deployment-charts@master] eventgate-logging values: update puppet CA

https://gerrit.wikimedia.org/r/554584

jbond updated the task description. (Show Details)
jbond removed a project: DBA.
  • 'trafficserver' restarted on all cache-ats nodes
  • puppetdb and postgresql restarted on puppetdb[12]002
  • postgresql restarted on netbox[12]001
  • stunnel4 service restarted on all service using rsync warped with stunnel
  • rsyslog restarted every where that has kafka shipping
  • uwsgi-netbox restarted on netbox servers other references are from systemd timers and no action needed
  • varnishkafka-webrequest and varnishkafka-eventlogging restarted on effected servers
  • no restart required
    • mysqld_exporter_config.py is ran by puppet
    • backup_mariadb.py ran by cron
    • check_mariadb.py ran by icinga
    • debmonitor::client either run as an apt hook or via cron

I have now updated the relevant CA files in the private repo