Provide centralized OCSP stapling as part of acme-chief backend duties
Description
Details
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Vgutierrez | T219765 Implement server-side OCSP stapling | |||
| Resolved | Vgutierrez | T232988 Use acme-chief provided OCSP stapling responses |
Event Timeline
Change 500397 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] tlsproxy: Allow update-ocsp-all writing in /etc/acmecerts
Change 500397 merged by Vgutierrez:
[operations/puppet@production] tlsproxy: Allow update-ocsp-all writing in /etc/acmecerts
Change 516604 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@master] x509: Expose the OCSP URI of a Certificate as a property
Change 529202 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@master] ocsp: Provide basic functionality to perform OCSP requests
Change 516604 merged by jenkins-bot:
[operations/software/acme-chief@master] x509: Expose the OCSP URI of a Certificate as a property
Change 529202 merged by jenkins-bot:
[operations/software/acme-chief@master] ocsp: Provide basic functionality to perform OCSP requests
Change 530464 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@master] ocsp: Allow to load an existing OCSPResponse from disk
Change 530465 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@master] acme_chief: Provide OCSP responses
Change 530548 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@master] ocsp: Provide basic test coverage
Change 530806 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@master] api: Allow acme-chief clients to fetch OCSP responses
Change 530464 merged by jenkins-bot:
[operations/software/acme-chief@master] ocsp: Allow to load an existing OCSPResponse from disk
Change 530548 merged by jenkins-bot:
[operations/software/acme-chief@master] ocsp: Provide basic test coverage
Change 530465 merged by jenkins-bot:
[operations/software/acme-chief@master] acme_chief: Provide OCSP responses
Change 530806 merged by jenkins-bot:
[operations/software/acme-chief@master] api: Allow acme-chief clients to fetch OCSP responses
Change 533856 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@master] Release 0.21
Change 533856 merged by Vgutierrez:
[operations/software/acme-chief@master] Release 0.21
Change 536006 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@debian] x509: Expose the OCSP URI of a Certificate as a property
Change 536007 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@debian] ocsp: Provide basic functionality to perform OCSP requests
Change 536008 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@debian] ocsp: Allow to load an existing OCSPResponse from disk
Change 536009 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@debian] ocsp: Provide basic test coverage
Change 536010 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@debian] acme_chief: Provide OCSP responses
Change 536011 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@debian] api: Allow acme-chief clients to fetch OCSP responses
Change 536012 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@debian] Release 0.21
Change 536006 merged by jenkins-bot:
[operations/software/acme-chief@debian] x509: Expose the OCSP URI of a Certificate as a property
Change 536007 merged by jenkins-bot:
[operations/software/acme-chief@debian] ocsp: Provide basic functionality to perform OCSP requests
Change 536008 merged by jenkins-bot:
[operations/software/acme-chief@debian] ocsp: Allow to load an existing OCSPResponse from disk
Change 536009 merged by jenkins-bot:
[operations/software/acme-chief@debian] ocsp: Provide basic test coverage
Change 536010 merged by jenkins-bot:
[operations/software/acme-chief@debian] acme_chief: Provide OCSP responses
Change 536011 merged by jenkins-bot:
[operations/software/acme-chief@debian] api: Allow acme-chief clients to fetch OCSP responses
Change 536012 merged by jenkins-bot:
[operations/software/acme-chief@debian] Release 0.21
Change 536015 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@debian] debian: Add release 0.21 to changelog
Change 536015 merged by Vgutierrez:
[operations/software/acme-chief@debian] debian: Add release 0.21 to changelog
Mentioned in SAL (#wikimedia-operations) [2019-09-12T07:45:36Z] <vgutierrez> uploaded acme-chief 0.21 to apt.wikimedia.org (buster) - T219765
Mentioned in SAL (#wikimedia-operations) [2019-09-12T08:22:54Z] <vgutierrez> upgrading to acme-chief 0.21 on acmechief-test instances - T219765
After upgrading acme-chief on acmechief-test1001, a tiny storm of OCSP requests was generated:
Sep 12 08:23:24 acmechief-test1001 acme-chief-backend[457]: Missing/invalid DNS zone updater CMD timeout, using the default one: 60.00
Sep 12 08:23:25 acmechief-test1001 acme-chief-backend[457]: Number of certificates per status: Counter({'VALID': 48})
Sep 12 08:23:25 acmechief-test1001 acme-chief-backend[457]: Starting main loop...
Sep 12 08:23:25 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate apt / ec-prime256v1
Sep 12 08:23:25 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for apt / ec-prime256v1
Sep 12 08:23:25 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate apt / rsa-2048
Sep 12 08:23:25 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for apt / rsa-2048
Sep 12 08:23:25 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate archiva / ec-prime256v1
Sep 12 08:23:25 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for archiva / ec-prime256v1
Sep 12 08:23:25 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate archiva / rsa-2048
Sep 12 08:23:25 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for archiva / rsa-2048
Sep 12 08:23:25 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate cloudelastic / ec-prime256v1
Sep 12 08:23:25 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for cloudelastic / ec-prime256v1
Sep 12 08:23:25 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate cloudelastic / rsa-2048
Sep 12 08:23:26 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for cloudelastic / rsa-2048
Sep 12 08:23:26 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate dumps / ec-prime256v1
Sep 12 08:23:26 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for dumps / ec-prime256v1
Sep 12 08:23:26 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate dumps / rsa-2048
Sep 12 08:23:26 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for dumps / rsa-2048
Sep 12 08:23:26 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate gerrit / ec-prime256v1
Sep 12 08:23:26 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for gerrit / ec-prime256v1
Sep 12 08:23:26 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate gerrit / rsa-2048
Sep 12 08:23:26 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for gerrit / rsa-2048
Sep 12 08:23:26 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate icinga / ec-prime256v1
Sep 12 08:23:26 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for icinga / ec-prime256v1
Sep 12 08:23:26 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate icinga / rsa-2048
Sep 12 08:23:26 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for icinga / rsa-2048
Sep 12 08:23:26 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate idp / ec-prime256v1
Sep 12 08:23:26 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for idp / ec-prime256v1
Sep 12 08:23:26 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate idp / rsa-2048
Sep 12 08:23:27 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for idp / rsa-2048
Sep 12 08:23:27 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate ldap / ec-prime256v1
Sep 12 08:23:27 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for ldap / ec-prime256v1
Sep 12 08:23:27 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate ldap / rsa-2048
Sep 12 08:23:27 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for ldap / rsa-2048
Sep 12 08:23:27 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate ldap-codfw1dev / ec-prime256v1
Sep 12 08:23:27 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for ldap-codfw1dev / ec-prime256v1
Sep 12 08:23:27 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate ldap-codfw1dev / rsa-2048
Sep 12 08:23:27 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for ldap-codfw1dev / rsa-2048
Sep 12 08:23:27 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate ldap-labtest / ec-prime256v1
Sep 12 08:23:27 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for ldap-labtest / ec-prime256v1
Sep 12 08:23:27 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate ldap-labtest / rsa-2048
Sep 12 08:23:27 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for ldap-labtest / rsa-2048
Sep 12 08:23:27 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate librenms / ec-prime256v1
Sep 12 08:23:28 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for librenms / ec-prime256v1
Sep 12 08:23:28 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate librenms / rsa-2048
Sep 12 08:23:28 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for librenms / rsa-2048
Sep 12 08:23:28 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate lists / ec-prime256v1
Sep 12 08:23:28 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for lists / ec-prime256v1
Sep 12 08:23:28 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate lists / rsa-2048
Sep 12 08:23:28 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for lists / rsa-2048
Sep 12 08:23:28 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate mirrors / ec-prime256v1
Sep 12 08:23:28 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for mirrors / ec-prime256v1
Sep 12 08:23:28 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate mirrors / rsa-2048
Sep 12 08:23:28 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for mirrors / rsa-2048
Sep 12 08:23:28 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate mx / ec-prime256v1
Sep 12 08:23:28 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for mx / ec-prime256v1
Sep 12 08:23:28 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate mx / rsa-2048
Sep 12 08:23:28 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for mx / rsa-2048
Sep 12 08:23:28 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate netbox / ec-prime256v1
Sep 12 08:23:29 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for netbox / ec-prime256v1
Sep 12 08:23:29 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate netbox / rsa-2048
Sep 12 08:23:29 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for netbox / rsa-2048
Sep 12 08:23:29 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate non-canonical-redirect-1 / ec-prime256v1
Sep 12 08:23:29 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for non-canonical-redirect-1 / ec-prime256v1
Sep 12 08:23:29 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate non-canonical-redirect-1 / rsa-2048
Sep 12 08:23:29 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for non-canonical-redirect-1 / rsa-2048
Sep 12 08:23:29 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate non-canonical-redirect-2 / ec-prime256v1
Sep 12 08:23:29 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for non-canonical-redirect-2 / ec-prime256v1
Sep 12 08:23:29 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate non-canonical-redirect-2 / rsa-2048
Sep 12 08:23:29 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for non-canonical-redirect-2 / rsa-2048
Sep 12 08:23:29 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate non-canonical-redirect-3 / ec-prime256v1
Sep 12 08:23:29 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for non-canonical-redirect-3 / ec-prime256v1
Sep 12 08:23:29 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate non-canonical-redirect-3 / rsa-2048
Sep 12 08:23:30 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for non-canonical-redirect-3 / rsa-2048
Sep 12 08:23:30 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate non-canonical-redirect-4 / ec-prime256v1
Sep 12 08:23:30 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for non-canonical-redirect-4 / ec-prime256v1
Sep 12 08:23:30 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate non-canonical-redirect-4 / rsa-2048
Sep 12 08:23:30 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for non-canonical-redirect-4 / rsa-2048
Sep 12 08:23:30 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate non-canonical-redirect-5 / ec-prime256v1
Sep 12 08:23:30 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for non-canonical-redirect-5 / ec-prime256v1
Sep 12 08:23:30 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate non-canonical-redirect-5 / rsa-2048
Sep 12 08:23:30 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for non-canonical-redirect-5 / rsa-2048
Sep 12 08:23:30 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate non-canonical-redirect-6 / ec-prime256v1
Sep 12 08:23:30 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for non-canonical-redirect-6 / ec-prime256v1
Sep 12 08:23:30 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate non-canonical-redirect-6 / rsa-2048
Sep 12 08:23:30 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for non-canonical-redirect-6 / rsa-2048
Sep 12 08:23:30 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate tendril / ec-prime256v1
Sep 12 08:23:30 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for tendril / ec-prime256v1
Sep 12 08:23:30 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate tendril / rsa-2048
Sep 12 08:23:31 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for tendril / rsa-2048
Sep 12 08:23:31 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate unified / ec-prime256v1
Sep 12 08:23:31 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for unified / ec-prime256v1
Sep 12 08:23:31 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate unified / rsa-2048
Sep 12 08:23:31 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for unified / rsa-2048
Sep 12 08:23:31 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate wikibase / ec-prime256v1
Sep 12 08:23:31 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for wikibase / ec-prime256v1
Sep 12 08:23:31 acmechief-test1001 acme-chief-backend[457]: Refreshing live OCSP response for certificate wikibase / rsa-2048
Sep 12 08:23:31 acmechief-test1001 acme-chief-backend[457]: live OCSP response refreshed successfully for wikibase / rsa-2048and everything looks good:
root@acmechief-test1001:~# openssl ocsp -respin /var/lib/acme-chief/certs/unified/live/rsa-2048.ocsp -issuer /var/lib/acme-chief/certs/unified/live/rsa-2048.chain.crt -text
OCSP Response Data:
OCSP Response Status: successful (0x0)
Response Type: Basic OCSP Response
Version: 1 (0x0)
Responder Id: CN = Fake LE Intermediate X1
Produced At: Sep 11 15:00:00 2019 GMT
Responses:
Certificate ID:
Hash Algorithm: sha1
Issuer Name Hash: C29C130A07D1FF36475F8766B701C13205DF6527
Issuer Key Hash: C0CC0346B95820CC5C7270F3E12ECB20A6F5683A
Serial Number: FA7F94D7577501D47CFCA058E7933A341C88
Cert Status: good
This Update: Sep 11 15:00:00 2019 GMT
Next Update: Sep 18 15:00:00 2019 GMT
Signature Algorithm: sha256WithRSAEncryption
05:dc:94:ca:a5:21:18:4d:8a:e0:93:2d:8c:65:13:10:43:2a:
85:58:df:f6:1d:7a:15:a7:f6:de:d0:f3:f7:45:12:7a:26:67:
eb:bb:73:ee:e2:1b:3b:86:f5:49:1d:c3:84:a0:34:82:a9:ba:
81:e4:4f:20:ba:a4:35:67:90:f1:ac:13:b4:b2:a6:6e:41:bf:
1e:33:ca:e1:eb:f9:d4:9b:3e:2e:2e:29:65:ca:8d:56:b7:ac:
2c:81:5b:15:a3:29:2b:5d:19:33:70:23:9b:f7:c7:dc:73:ac:
73:eb:22:09:d6:8b:1c:6d:96:7f:65:ea:fe:12:bd:d0:2b:ba:
2d:77:12:1a:8e:d1:63:fe:80:7a:ec:3d:0f:d1:c2:84:d2:c5:
7a:b7:4c:bf:91:01:3c:4d:2f:c8:2b:d8:7d:c9:99:1e:ac:76:
8f:c9:21:eb:77:75:cd:a1:4c:4f:8b:53:95:35:5d:06:57:b6:
60:e4:d6:8f:e3:ae:ce:27:0d:c3:de:41:a1:72:5d:29:fc:bc:
6b:e0:f7:d4:03:c8:53:19:85:90:a6:2c:aa:62:c5:e8:c1:2b:
6a:19:7a:df:02:d1:f9:ac:71:dc:7c:4c:27:0d:e1:21:62:70:
46:17:3a:8f:5a:97:46:a1:96:4f:8d:f4:24:af:ac:54:84:46:
87:aa:4a:26
Response verify OKMentioned in SAL (#wikimedia-operations) [2019-09-16T10:16:36Z] <vgutierrez> upgrade acme-chief production servers to acme-chief 0.21 - T219765