Page MenuHomePhabricator

labvirt boxes need a new cert for libvirtd
Closed, ResolvedPublic

Description

On virt10xx, libvirtd uses a WMF CA-signed wildcard cert for virt*.eqiad.wmnet.

As part of the renaming from virt* to labvirt* we need a new cert for labvirt*.eqiad.wmnet.

https://gerrit.wikimedia.org/r/#/c/204279/ is an attempt to do this, but that cert is self-signed and we need one signed with the WMF CA.

In the meantime, libvirtd.log has sad, repeated notices:

error : virNetTLSContextCheckCertPair:495 : Our own certificate /etc/ssl/localcerts/labvirt-star.eqiad.wmnet.crt failed validation against /etc/ssl/certs/wmf-ca.pem: The certificate hasn't got a known issuer.

Event Timeline

Andrew assigned this task to akosiaris.
Andrew raised the priority of this task from to Needs Triage.
Andrew updated the task description. (Show Details)
Andrew added a project: Cloud-Services.
Andrew subscribed.

So, there are 2 WMF CAs right now. The "old" one and the "new" one. Those are populated on all systems via

https://github.com/wikimedia/operations-puppet/blob/production/manifests/certs.pp#L69

and

https://github.com/wikimedia/operations-puppet/blob/production/manifests/certs.pp#L75

respectively.

We are phasing out the old one, so no more certs will be issued from it. The new one will be used instead

Change 204612 had a related patch set uploaded (by Alexandros Kosiaris):
Populate labvirtstar from wmf_ca_2014_2017

https://gerrit.wikimedia.org/r/204612

Change 204612 merged by Andrew Bogott:
Populate labvirtstar from wmf_ca_2014_2017

https://gerrit.wikimedia.org/r/204612

libvirtd.conf only allows me to specify one ca file. So I either need keys on all boxes from the same CA, or some way to chain multiple CAs in a single file.

It shouldn't hurt to replace the cert on the existing virt10xx boxes. Alex, do you mind generating me new virt-star certs that also use the new CA?

thanks!

Oh, I should add -- the main project here is migration of instances from virt10xx to labvirt10xx. So I need them to talk to each other.

Change 204718 had a related patch set uploaded (by Alexandros Kosiaris):
Issue new certificate for virt-star

https://gerrit.wikimedia.org/r/204718

OK, change uploaded https://gerrit.wikimedia.org/r/#/c/204718/. I am unsure of the repercussions of this one. Probably needs to be coordinated so that it happens on all virt nodes in the same time followed by a restart of libvirtd. It should not have any repercussions on running VMs though

/me happy the old ca goes away. only opendj is left now.

Change 204718 merged by Andrew Bogott:
Issue new certificate for virt-star

https://gerrit.wikimedia.org/r/204718

Certs successfully issued from privated CA and populated on servers, resolving