The TLS certificate for https://wma.wmflabs.org/ expired on 9/7/2020, 6:03:09 PM (Eastern Daylight Time). This tool is linked from any Wikipedia article with coordinates, making it highly visible. It's been expired for 5 hours and one complaint has already made it to IRC.
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | aborrero | T262237 The TLS certificate for https://wma.wmflabs.org is expired | |||
| Open | Vgutierrez | T262251 acme-chief shouldn't try to perform OCSP stapling of expired certs |
Event Timeline
Comment Actions
Mentioned in SAL (#wikimedia-cloud) [2020-09-08T09:10:34Z] <arturo> restart acme-chief service in roject-proxy-acme-chief-01 (T262237)
Comment Actions
Mentioned in SAL (#wikimedia-cloud) [2020-09-08T09:18:07Z] <arturo> upgrading acme-chief deb package from 0.25-1 to 0.28-1 on project-proxy-acme-chief-01 (T262237)
Comment Actions
The acme-chief backend gets OCSPResponseStatus.UNAUTHORIZED and can't generate the new certs apparently.
Sep 08 09:29:24 project-proxy-acme-chief-01 acme-chief-backend[15987]: Waiting till tiles / ec-prime256v1 is generated to be able to push the new certificate
Sep 08 09:29:24 project-proxy-acme-chief-01 acme-chief-backend[15987]: Traceback (most recent call last):
Sep 08 09:29:24 project-proxy-acme-chief-01 acme-chief-backend[15987]: File "/usr/bin/acme-chief-backend", line 11, in <module>
Sep 08 09:29:24 project-proxy-acme-chief-01 acme-chief-backend[15987]: load_entry_point('acme-chief==0.28', 'console_scripts', 'acme-chief-backend')()
Sep 08 09:29:24 project-proxy-acme-chief-01 acme-chief-backend[15987]: File "/usr/lib/python3/dist-packages/acme_chief/acme_chief.py", line 928, in main
Sep 08 09:29:24 project-proxy-acme-chief-01 acme-chief-backend[15987]: ACMEChief().run()
Sep 08 09:29:24 project-proxy-acme-chief-01 acme-chief-backend[15987]: File "/usr/lib/python3/dist-packages/acme_chief/acme_chief.py", line 377, in run
Sep 08 09:29:24 project-proxy-acme-chief-01 acme-chief-backend[15987]: self.certificate_management()
Sep 08 09:29:24 project-proxy-acme-chief-01 acme-chief-backend[15987]: File "/usr/lib/python3/dist-packages/acme_chief/acme_chief.py", line 906, in certificate_management
Sep 08 09:29:24 project-proxy-acme-chief-01 acme-chief-backend[15987]: self._fetch_ocsp_response(cert_id, key_type_id)
Sep 08 09:29:24 project-proxy-acme-chief-01 acme-chief-backend[15987]: File "/usr/lib/python3/dist-packages/acme_chief/acme_chief.py", line 849, in _fetch_ocsp_response
Sep 08 09:29:24 project-proxy-acme-chief-01 acme-chief-backend[15987]: if ocsp_response.next_update - datetime.datetime.utcnow() < cert_details['ocsp_update_threshold']:
Sep 08 09:29:24 project-proxy-acme-chief-01 acme-chief-backend[15987]: File "/usr/lib/python3/dist-packages/acme_chief/ocsp.py", line 105, in next_update
Sep 08 09:29:24 project-proxy-acme-chief-01 acme-chief-backend[15987]: return self._response.next_update
Sep 08 09:29:24 project-proxy-acme-chief-01 acme-chief-backend[15987]: File "/usr/lib/python3/dist-packages/cryptography/hazmat/backends/openssl/ocsp.py", line 30, in wrapper
Sep 08 09:29:24 project-proxy-acme-chief-01 acme-chief-backend[15987]: "OCSP response status is not successful so the property "
Sep 08 09:29:24 project-proxy-acme-chief-01 acme-chief-backend[15987]: ValueError: OCSP response status is not successful so the property has no valueBut this is for the 'tiles' certificate set, and not for wma? weird.
Comment Actions
Mentioned in SAL (#wikimedia-cloud) [2020-09-08T10:05:52Z] <arturo> remove /var/lib/acme-chief/certs/* to force acme-chief generating new certs instead of renewing them (T262237)
Comment Actions
This should be fixed now. Thanks @Vgutierrez for the assistance on IRC.
Closing task now, thanks @AntiCompositeNumber feel free to reopen if required.