Page MenuHomePhabricator
Create Task
Maniphest T255249

acme-chief: support for generating a concatenated cert/key file
Closed, ResolvedPublic
Actions

Description

For some uses cases, it is interesting to have a concatenated cert/key file. This single file is mandatory in some services, like haproxy in version 1.8. We are using haproxy with TLS in T195217: Simplify ingress methods for PAWS

We don't have any puppet code to generate such file, which however should be rather easy to generate with something like $ cat file.cert file.key > concat.pem.
The generated file should be regenerated when the source original cert files change, and the generated filed should notify same puppet resources as the original cert files.

Details

Show related patches Customize query in gerrit

Related Objects

Event Timeline

aborrero created this task.Jun 12 2020, 10:37 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 12 2020, 10:37 AM
Vgutierrez triaged this task as Medium priority.Jun 12 2020, 10:38 AM
gerritbot added a comment.Jun 12 2020, 2:17 PM

Change 605237 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@master] acme_chief,x509: Provide .crt.key file support

https://gerrit.wikimedia.org/r/605237

gerritbot added a project: Patch-For-Review.Jun 12 2020, 2:17 PM
gerritbot added a comment.Jun 12 2020, 3:14 PM

Change 605254 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@master] api: Allow acme-chief clients to fetch .crt.key files

https://gerrit.wikimedia.org/r/605254

Krenair subscribed.Jun 12 2020, 7:32 PM
gerritbot added a comment.Jun 15 2020, 12:25 PM

Change 605237 merged by Vgutierrez:
[operations/software/acme-chief@master] acme_chief,x509: Provide .crt.key file support

https://gerrit.wikimedia.org/r/605237

gerritbot added a comment.Jun 15 2020, 12:26 PM

Change 605254 merged by Vgutierrez:
[operations/software/acme-chief@master] api: Allow acme-chief clients to fetch .crt.key files

https://gerrit.wikimedia.org/r/605254

gerritbot added a comment.Jun 15 2020, 12:29 PM

Change 605577 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@master] Release 0.26

https://gerrit.wikimedia.org/r/605577

gerritbot added a comment.Jun 15 2020, 12:32 PM

Change 605577 merged by Vgutierrez:
[operations/software/acme-chief@master] Release 0.26

https://gerrit.wikimedia.org/r/605577

gerritbot added a comment.Jun 15 2020, 12:35 PM

Change 605579 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@debian] acme_chief,x509: Provide .crt.key file support

https://gerrit.wikimedia.org/r/605579

gerritbot added a comment.Jun 15 2020, 12:35 PM

Change 605580 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@debian] api: Allow acme-chief clients to fetch .crt.key files

https://gerrit.wikimedia.org/r/605580

gerritbot added a comment.Jun 15 2020, 12:35 PM

Change 605581 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@debian] Release 0.26

https://gerrit.wikimedia.org/r/605581

gerritbot added a comment.Jun 15 2020, 12:37 PM

Change 605582 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@debian] debian: Add release 0.26 to changelog

https://gerrit.wikimedia.org/r/605582

gerritbot added a comment.Jun 15 2020, 12:41 PM

Change 605579 merged by Vgutierrez:
[operations/software/acme-chief@debian] acme_chief,x509: Provide .crt.key file support

https://gerrit.wikimedia.org/r/605579

gerritbot added a comment.Jun 15 2020, 12:41 PM

Change 605580 merged by Vgutierrez:
[operations/software/acme-chief@debian] api: Allow acme-chief clients to fetch .crt.key files

https://gerrit.wikimedia.org/r/605580

gerritbot added a comment.Jun 15 2020, 12:41 PM

Change 605581 merged by Vgutierrez:
[operations/software/acme-chief@debian] Release 0.26

https://gerrit.wikimedia.org/r/605581

gerritbot added a comment.Jun 15 2020, 12:42 PM

Change 605582 merged by Vgutierrez:
[operations/software/acme-chief@debian] debian: Add release 0.26 to changelog

https://gerrit.wikimedia.org/r/605582

Stashbot added a comment.Jun 15 2020, 12:46 PM

Mentioned in SAL (#wikimedia-operations) [2020-06-15T12:46:04Z] <vgutierrez> upload acme-chief 0.26 to apt.wm.o (buster) - T255249

Vgutierrez closed this task as Resolved.Jun 15 2020, 12:57 PM

This seems to be working (from my tests on acmechief-test1001):

root@acmechief-test1001:/var/lib/acme-chief/certs/mirrors/new# grep "BEGIN EC PRIVATE KEY" ec-prime256v1.crt.key
-----BEGIN EC PRIVATE KEY-----
root@acmechief-test1001:/var/lib/acme-chief/certs/mirrors/new# grep "BEGIN CERT" ec-prime256v1.crt.key
-----BEGIN CERTIFICATE-----

Please do not forget to restart acme-chief and uwsgi-acme-chief after upgrading to version 0.26. This change will only be visible on certs reissued after the upgrade to 0.26.

Maintenance_bot removed a project: Patch-For-Review.Jun 15 2020, 1:11 PM
aborrero awarded a token.Jun 15 2020, 3:56 PM
bd808 subscribed.Jul 29 2020, 4:06 PM

It turns out that we really need a .crt.chained.key version of this that includes the LE signed cert, the local key, and the intermediate signing chain.

$ curl -v https://public.paws.wmcloud.org/
*   Trying 185.15.56.57...
* TCP_NODELAY set
* Connected to public.paws.wmcloud.org (185.15.56.57) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS alert, Server hello (2):
* SSL certificate problem: unable to get local issuer certificate
* Curl_http_done: called premature == 1
* stopped the pause stream!
* Closing connection 0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html
gerritbot added a comment.Jul 29 2020, 4:53 PM

Change 617176 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[operations/software/acme-chief@master] acme_chief: Profide .crt.chained.key file support

https://gerrit.wikimedia.org/r/617176

gerritbot added a project: Patch-For-Review.Jul 29 2020, 4:53 PM

Change 617177 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[operations/software/acme-chief@master] api: Allow acme-chief clients to fetch .chained.crt.key files

https://gerrit.wikimedia.org/r/617177

bd808 reopened this task as Open.Jul 29 2020, 4:59 PM
gerritbot added a comment.Jul 30 2020, 9:02 AM

Change 617176 merged by jenkins-bot:
[operations/software/acme-chief@master] acme_chief: Profide .chained.crt.key file support

https://gerrit.wikimedia.org/r/617176

gerritbot added a comment.Jul 30 2020, 9:02 AM

Change 617177 merged by jenkins-bot:
[operations/software/acme-chief@master] api: Allow acme-chief clients to fetch .chained.crt.key files

https://gerrit.wikimedia.org/r/617177

gerritbot added a comment.Jul 30 2020, 9:12 AM

Change 617400 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@master] Release 0.27

https://gerrit.wikimedia.org/r/617400

gerritbot added a comment.Jul 30 2020, 9:32 AM

Change 617400 merged by jenkins-bot:
[operations/software/acme-chief@master] Release 0.27

https://gerrit.wikimedia.org/r/617400

gerritbot added a comment.Jul 30 2020, 12:32 PM

Change 617424 had a related patch set uploaded (by Vgutierrez; owner: Bryan Davis):
[operations/software/acme-chief@debian] acme_chief: Profide .chained.crt.key file support

https://gerrit.wikimedia.org/r/617424

gerritbot added a comment.Jul 30 2020, 12:32 PM

Change 617425 had a related patch set uploaded (by Vgutierrez; owner: Bryan Davis):
[operations/software/acme-chief@debian] api: Allow acme-chief clients to fetch .chained.crt.key files

https://gerrit.wikimedia.org/r/617425

gerritbot added a comment.Jul 30 2020, 12:32 PM

Change 617446 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@debian] Release 0.27

https://gerrit.wikimedia.org/r/617446

gerritbot added a comment.Jul 30 2020, 12:32 PM

Change 617447 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@debian] debian: Add release 0.27 to changelog

https://gerrit.wikimedia.org/r/617447

gerritbot added a comment.Jul 30 2020, 12:40 PM

Change 617424 merged by jenkins-bot:
[operations/software/acme-chief@debian] acme_chief: Profide .chained.crt.key file support

https://gerrit.wikimedia.org/r/617424

gerritbot added a comment.Jul 30 2020, 12:40 PM

Change 617425 merged by jenkins-bot:
[operations/software/acme-chief@debian] api: Allow acme-chief clients to fetch .chained.crt.key files

https://gerrit.wikimedia.org/r/617425

gerritbot added a comment.Jul 30 2020, 12:40 PM

Change 617446 merged by jenkins-bot:
[operations/software/acme-chief@debian] Release 0.27

https://gerrit.wikimedia.org/r/617446

gerritbot added a comment.Jul 30 2020, 1:42 PM

Change 617447 merged by jenkins-bot:
[operations/software/acme-chief@debian] debian: Add release 0.27 to changelog

https://gerrit.wikimedia.org/r/617447

Stashbot added a comment.Jul 30 2020, 1:47 PM

Mentioned in SAL (#wikimedia-operations) [2020-07-30T13:47:01Z] <vgutierrez> upload acme-chief 0.27 to apt.wm.o (buster) - T255249

Stashbot added a comment.Jul 30 2020, 1:47 PM

Mentioned in SAL (#wikimedia-operations) [2020-07-30T13:47:25Z] <vgutierrez> upgrade acme-chief to version 0.27 - T255249

Vgutierrez added a comment.Jul 30 2020, 1:54 PM

@bd808 acme-chief 0.27 shipping your changes has been deployed in production. Please note that your change will be effective the next time acme-chief reissues your cert.

Stashbot added a comment.Jul 30 2020, 4:38 PM

Mentioned in SAL (#wikimedia-cloud) [2020-07-30T16:38:50Z] <bstorm> removing the *.paws.wmflabs.org SNI name because it won't be used and it might trigger a re-issue of certs T255249

• Bstorm subscribed.Jul 30 2020, 4:43 PM

It reissued, but I'm not seeing the new ec-prime256v1.chained.crt.key on the client, at least with what puppet grabbed. Digging a bit there.

• Bstorm added a comment.Jul 30 2020, 4:45 PM

Ah, I see, it's the "live" link vs. the "new" link.

• Bstorm added a comment.EditedJul 30 2020, 4:48 PM

That's strange. I only have rsa certs, not ecc ones from the latest run.

root@paws-k8s-haproxy-1:/etc/acmecerts/paws# ls live
ec-prime256v1.chain.crt    ec-prime256v1.crt	  ec-prime256v1.key   rsa-2048.chain.crt    rsa-2048.crt      rsa-2048.key
ec-prime256v1.chained.crt  ec-prime256v1.crt.key  ec-prime256v1.ocsp  rsa-2048.chained.crt  rsa-2048.crt.key  rsa-2048.ocsp
root@paws-k8s-haproxy-1:/etc/acmecerts/paws# ls new
ec-prime256v1.key  rsa-2048.chain.crt  rsa-2048.chained.crt  rsa-2048.chained.crt.key  rsa-2048.crt  rsa-2048.crt.key  rsa-2048.key  rsa-2048.ocsp

So there is a rsa-2048.chained.crt.key but there is no ec-prime256v1.chained.crt.key.

• Bstorm added a comment.Jul 30 2020, 4:51 PM

They have appeared! There's a delay on the ec-prime256 certs

• Bstorm added a comment.Jul 30 2020, 4:53 PM

And that updated the "live" link

gerritbot added a comment.Jul 30 2020, 5:08 PM

Change 617497 had a related patch set uploaded (by Bstorm; owner: Bstorm):
[operations/puppet@production] paws haproxy: switch to the chained cert for TLS

https://gerrit.wikimedia.org/r/617497

gerritbot added a comment.Jul 30 2020, 5:10 PM

Change 617497 merged by Bstorm:
[operations/puppet@production] paws haproxy: switch to the chained cert for TLS

https://gerrit.wikimedia.org/r/617497

• Bstorm closed this task as Resolved.Jul 30 2020, 5:13 PM

It works!

Krenair added a comment.Jul 30 2020, 5:47 PM

I think the keys are generated first and the certs appear when acme-chief
has gone through the ACME API to get stuff signed by the CA

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits