Page MenuHomePhabricator

Update puppetmaster1001 puppet certificate
Closed, ResolvedPublic


The puppetmaster1001.eqiad.wmnet puppet certificate was revoked some time ago for an unknown reason. An updated certificate needs to be created very carefully.

Event Timeline

Puppetmaster1001 is not only a puppet master but the ca server so we need to be very cautious.

Typically recreating an agent cert is along these lines:

  1. on agent: puppet agent --disable "regenerating cert"
  2. on agent: mv /var/lib/puppet/ssl /var/lib/puppet/ssl.orig
  3. on master: puppet cert clean fqdn_of_agent
  4. on agent: puppet agent --enable
  5. on agent: puppet agent -t
  6. on master: puppet cert sign fqdn_of_agent

However, on puppetmaster1001 there are additional private keys in /var/lib/puppet/ssl/private_keys as well as the /var/lib/puppet/ssl/certs/puppet.pem cert, so we need to be more precise and move only the revoked cert aside. It's also both the agent and master in this case. So something like this (all commands on puppetmaster1001)

  1. puppet agent --disable "regenerating cert"
  2. cp -a /var/lib/puppet/ssl /var/lib/puppet/ssl.bak-$date
  3. cp -a /var/lib/puppet/server /var/lib/puppet/server.bak-$date (paranoia)
  4. mv /var/lib/puppet/ssl/certs/ca.pem /var/lib/puppet/ssl/certs/ca.pem.bak-$date
  5. mv /var/lib/puppet/ssl/certs/puppetmaster1001.eqiad.wmnet.pem /var/lib/puppet/ssl/certs/puppetmaster1001.eqiad.wmnet.pem.bak-$date
  6. mv /var/lib/puppet/ssl/private_keys/puppetmaster1001.eqiad.wmnet.pem /var/lib/puppet/ssl/private_keys/puppetmaster1001.eqiad.wmnet.pem.bak-$date
  7. mv /var/lib/puppet/ssl/public_keys/puppetmaster1001.eqiad.wmnet.pem /var/lib/puppet/ssl/public_keys/puppetmaster1001.eqiad.wmnet.pem.bak-$date
  8. puppet cert clean puppetmaster1001.eqiad.wmnet
  9. puppet agent --enable
  10. puppet agent -t
  11. puppet cert sign puppetmaster1001.eqiad.wmnet
herron triaged this task as High priority.Nov 9 2017, 9:17 PM
herron added a subscriber: Joe.

To reduce risk I think we should tackle this after depooling the eqiad puppet masters for upgrades

herron claimed this task.