Page MenuHomePhabricator

toolforge: certificate issues in both tools-cdnproxy-0x.eqiad.wmflabs
Closed, ResolvedPublic

Description

Shinken reported that both servers tools-cdnproxy-01.eqiad.wmflabs and tools-cdnproxy-02.eqiad.wmflabs have issues with puppet.

The error is the same in both nodes:

aborrero@tools-cdnproxy-01:~$ sudo puppet agent -t -v
Warning: Unable to fetch my node definition, but the agent run will continue:
Warning: SSL_connect returned=1 errno=0 state=error: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet CA: tools-puppetmaster-01.tools.eqiad.wmflabs]
Info: Retrieving pluginfacts
Error: /File[/var/lib/puppet/facts.d]: Failed to generate additional resources using 'eval_generate': SSL_connect returned=1 errno=0 state=error: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet CA: tools-puppetmaster-01.tools.eqiad.wmflabs]
Error: /File[/var/lib/puppet/facts.d]: Could not evaluate: Could not retrieve file metadata for puppet:///pluginfacts: SSL_connect returned=1 errno=0 state=error: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet CA: tools-puppetmaster-01.tools.eqiad.wmflabs]
Info: Retrieving plugin
Error: /File[/var/lib/puppet/lib]: Failed to generate additional resources using 'eval_generate': SSL_connect returned=1 errno=0 state=error: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet CA: tools-puppetmaster-01.tools.eqiad.wmflabs]
Error: /File[/var/lib/puppet/lib]: Could not evaluate: Could not retrieve file metadata for puppet:///plugins: SSL_connect returned=1 errno=0 state=error: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet CA: tools-puppetmaster-01.tools.eqiad.wmflabs]
Info: Loading facts
Error: Could not retrieve catalog from remote server: SSL_connect returned=1 errno=0 state=error: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet CA: tools-puppetmaster-01.tools.eqiad.wmflabs]
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run
Error: Could not send report: SSL_connect returned=1 errno=0 state=error: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet CA: tools-puppetmaster-01.tools.eqiad.wmflabs]

Event Timeline

Oops, sorry. I just discovered these are new instances created by @Bstorm the other day.

Yup. I have to figure out the next step on those. Sorry if they caused trouble.

I think your need to regenerate the ssl cert by doing

rm -rf /var/lib/puppet/ssl

then puppet agent -tv

then sign the cert on the tools puppet master.

Yup. I have to figure out the next step on those. Sorry if they caused trouble.

No problem :-) My fault.

Got the agents set up and syncing correctly.