I haven't investigated this yet but am creating this task for tracking purposes.
Assuming that paws is using acme-chief this probably just requires a service restart.
Description
Description
Related Objects
Related Objects
- Mentioned In
- T328842: Restructure paws away from special networking
- Mentioned Here
- T307333: [tools] toolserver.org cert is expiring in 2 days
Event Timeline
Comment Actions
Mentioned in SAL (#wikimedia-cloud) [2022-05-14T16:16:57Z] <andrewbogott> restarting acme-chief.service on paws-acme-chief-01 for T308383
Comment Actions
Thank you! I think the restart did it:
openssl s_client -servername paws.wmcloud.org -connect paws.wmcloud.org:443 2>/dev/null | openssl x509 -noout -dates notBefore=May 14 15:16:50 2022 GMT notAfter=Aug 12 15:16:49 2022 GMT
Looks like we're good until August.
Comment Actions
This should not be needed, the service has a set of timers that should be triggering the config updates instead, maybe that is not working anymore, see T307333: [tools] toolserver.org cert is expiring in 2 days
Comment Actions
yep, that seems to be the case (see that NEXT is n/a):
root@paws-acme-chief-01:~# systemctl list-timers NEXT LEFT LAST PASSED UNIT ACTIVATES ... n/a n/a Mon 2021-11-22 23:59:14 UTC 5 months 22 days ago reload-acme-chief-backend.timer reload-acme-chief-backend.service ...
It's not clear why (a race condition with the timer + when puppet changed it) but restarting it works, and it should be up again the next time the VM reboots too:
root@paws-acme-chief-01:~# systemctl restart reload-acme-chief-backend.service ... root@paws-acme-chief-01:~# systemctl list-timers NEXT LEFT LAST PASSED UNIT ACTIVATES ... Mon 2022-05-16 10:22:20 UTC 59min left Mon 2022-05-16 09:22:20 UTC 30s ago reload-acme-chief-backend.timer reload-acme-chief-backend.service