Page MenuHomePhabricator

acme-chief is unable to renew certificates against LE staging environment
Closed, ResolvedPublic

Description

acme-chief fails in acmechief-test1001 to renew certificates, the log shows the following error:

Feb 04 13:44:20 acmechief-test1001 acme-chief-backend[28213]: Handling order finalized event for apt / ec-prime256v1
Feb 04 13:44:21 acmechief-test1001 acme-chief-backend[28213]: Traceback (most recent call last):
Feb 04 13:44:21 acmechief-test1001 acme-chief-backend[28213]:   File "/usr/bin/acme-chief-backend", line 11, in <module>
Feb 04 13:44:21 acmechief-test1001 acme-chief-backend[28213]:     load_entry_point('acme-chief==0.22', 'console_scripts', 'acme-chief-backend')()
Feb 04 13:44:21 acmechief-test1001 acme-chief-backend[28213]:   File "/usr/lib/python3/dist-packages/acme_chief/acme_chief.py", line 908, in main
Feb 04 13:44:21 acmechief-test1001 acme-chief-backend[28213]:     ACMEChief().run()
Feb 04 13:44:21 acmechief-test1001 acme-chief-backend[28213]:   File "/usr/lib/python3/dist-packages/acme_chief/acme_chief.py", line 364, in run
Feb 04 13:44:21 acmechief-test1001 acme-chief-backend[28213]:     self.certificate_management()
Feb 04 13:44:21 acmechief-test1001 acme-chief-backend[28213]:   File "/usr/lib/python3/dist-packages/acme_chief/acme_chief.py", line 881, in certificate_management
Feb 04 13:44:21 acmechief-test1001 acme-chief-backend[28213]:     new_status = self._handle_pushed_challenges(cert_id, key_type_id)
Feb 04 13:44:21 acmechief-test1001 acme-chief-backend[28213]:   File "/usr/lib/python3/dist-packages/acme_chief/acme_chief.py", line 687, in _handle_pushed_challenges
Feb 04 13:44:21 acmechief-test1001 acme-chief-backend[28213]:     status = self._handle_order_finalized(cert_id, key_type_id)
Feb 04 13:44:21 acmechief-test1001 acme-chief-backend[28213]:   File "/usr/lib/python3/dist-packages/acme_chief/acme_chief.py", line 715, in _handle_order_finalized
Feb 04 13:44:21 acmechief-test1001 acme-chief-backend[28213]:     certificate = session.get_certificate(csr_id)
Feb 04 13:44:21 acmechief-test1001 acme-chief-backend[28213]:   File "/usr/lib/python3/dist-packages/acme_chief/acme_requests.py", line 483, in get_certificate
Feb 04 13:44:21 acmechief-test1001 acme-chief-backend[28213]:     certificate_order = self.acme_client.fetch_certificate(finished_order, deadline=deadline)
Feb 04 13:44:21 acmechief-test1001 acme-chief-backend[28213]:   File "/usr/lib/python3/dist-packages/acme_chief/acme_requests.py", line 227, in fetch_certificate
Feb 04 13:44:21 acmechief-test1001 acme-chief-backend[28213]:     response = self.net.get(orderr.uri)
Feb 04 13:44:21 acmechief-test1001 acme-chief-backend[28213]:   File "/usr/lib/python3/dist-packages/acme/client.py", line 1171, in get
Feb 04 13:44:21 acmechief-test1001 acme-chief-backend[28213]:     self._send_request('GET', url, **kwargs), content_type=content_type)
Feb 04 13:44:21 acmechief-test1001 acme-chief-backend[28213]:   File "/usr/lib/python3/dist-packages/acme/client.py", line 1073, in _check_response
Feb 04 13:44:21 acmechief-test1001 acme-chief-backend[28213]:     raise messages.Error.from_json(jobj)
Feb 04 13:44:21 acmechief-test1001 acme-chief-backend[28213]: acme.messages.Error: urn:ietf:params:acme:error:malformed :: The request message was malformed :: Method not allowed

This could be related to the python3-acme version we are currently using and some recent changes on Let's Encrypt regarding POST-as-get requests

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Vgutierrez moved this task from Backlog to TLS on the Traffic board.

Change 570252 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@master] requests: Use POST-as-GET to fetch the issued certificate

https://gerrit.wikimedia.org/r/570252

Change 570252 merged by Vgutierrez:
[operations/software/acme-chief@master] requests: Use POST-as-GET to fetch the issued certificate

https://gerrit.wikimedia.org/r/570252

Change 570303 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@master] Release 0.23

https://gerrit.wikimedia.org/r/570303

Change 570303 merged by Vgutierrez:
[operations/software/acme-chief@master] Release 0.23

https://gerrit.wikimedia.org/r/570303

Change 570307 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@debian] requests: Use POST-as-GET to fetch the issued certificate

https://gerrit.wikimedia.org/r/570307

Change 570308 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@debian] Release 0.23

https://gerrit.wikimedia.org/r/570308

Change 570309 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@debian] debian: Add release 0.23 to changelog

https://gerrit.wikimedia.org/r/570309

Change 570307 merged by jenkins-bot:
[operations/software/acme-chief@debian] requests: Use POST-as-GET to fetch the issued certificate

https://gerrit.wikimedia.org/r/570307

Change 570308 merged by jenkins-bot:
[operations/software/acme-chief@debian] Release 0.23

https://gerrit.wikimedia.org/r/570308

Change 570309 merged by Vgutierrez:
[operations/software/acme-chief@debian] debian: Add release 0.23 to changelog

https://gerrit.wikimedia.org/r/570309

Mentioned in SAL (#wikimedia-operations) [2020-02-05T13:16:11Z] <vgutierrez> upload acme-chief 0.23 to apt.wm.o (buster) - T244236

Change 570332 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@master] requests: Fix content-type on fetch_certificate

https://gerrit.wikimedia.org/r/570332

Change 570332 merged by Vgutierrez:
[operations/software/acme-chief@master] requests: Fix content-type on fetch_certificate

https://gerrit.wikimedia.org/r/570332

Change 570338 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@master] Release 0.24

https://gerrit.wikimedia.org/r/570338

Change 570338 merged by jenkins-bot:
[operations/software/acme-chief@master] Release 0.24

https://gerrit.wikimedia.org/r/570338

Change 570340 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@debian] requests: Fix content-type on fetch_certificate

https://gerrit.wikimedia.org/r/570340

Change 570341 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@debian] Release 0.24

https://gerrit.wikimedia.org/r/570341

Change 570342 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/software/acme-chief@debian] debian: Add release 0.24 to changelog

https://gerrit.wikimedia.org/r/570342

Change 570340 merged by Vgutierrez:
[operations/software/acme-chief@debian] requests: Fix content-type on fetch_certificate

https://gerrit.wikimedia.org/r/570340

Change 570341 merged by Vgutierrez:
[operations/software/acme-chief@debian] Release 0.24

https://gerrit.wikimedia.org/r/570341

Change 570342 merged by Vgutierrez:
[operations/software/acme-chief@debian] debian: Add release 0.24 to changelog

https://gerrit.wikimedia.org/r/570342

Mentioned in SAL (#wikimedia-operations) [2020-02-05T14:30:30Z] <vgutierrez> upload acme-chief 0.24 to apt.wm.o (buster) - T244236

Mentioned in SAL (#wikimedia-operations) [2020-02-05T14:34:59Z] <vgutierrez> updating acme-chief to version 0.24 - T244236