Page MenuHomePhabricator
Create Task
Maniphest T171157

Monitor internal CA expirations
Closed, DeclinedPublic
Actions

Description

Yesterday we had the WMF CA 2014-2017 expire without any kind of warning, causing the handful of certificates issued by it to expire, which in turn cascaded into a widespread monitoring tools and WMCS outage. We should monitor for the expiration of that CA, and warn sufficiently in advance in case of expiry.

Side note: I renewed the CA for another 3 years (2017-2020) rather than e.g. 10 on purpose, to make sure that these CA refreshing procedures are exercised often. I'd be inclined to make it even shorter.

While at it, we should also make sure other internal CAs are monitored as well. The Puppet CA, which is used more as a general purpose CA these days, also immediately to mind.

Related Objects

Event Timeline

faidon created this task.Jul 20 2017, 10:03 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 20 2017, 10:03 AM
faidon edited projects, added observability; removed community-labs-monitoring.Jul 20 2017, 10:03 AM
faidon moved this task from Inbox to Up next on the observability board.
Paladox subscribed.Jul 20 2017, 11:00 AM
faidon assigned this task to Dzahn.Jul 24 2017, 3:17 PM
faidon changed the task status from Open to Stalled.Aug 21 2017, 3:13 PM

Setting to stalled until we decide what to actually do with the internal CA, as we're considering dropping it entirely in favour of other options.

faidon reassigned this task from Dzahn to akosiaris.Sep 18 2017, 3:19 PM
faidon added a subscriber: Dzahn.
akosiaris moved this task from Up next to Inbox on the observability board.Jan 8 2018, 4:36 PM
• Phabricator_maintenance moved this task from Backlog to Acknowledged on the SRE board.Jan 26 2019, 9:23 PM
fgiunchedi moved this task from Inbox to Radar on the observability board.Jul 20 2020, 1:16 PM
Aklapper added a comment.Nov 9 2020, 12:22 AM
In T171157#3538015, @faidon wrote:

Setting to stalled until we decide what to actually do with the internal CA, as we're considering dropping it entirely in favour of other options.

@akosiaris / @faidon: Has this situation somehow changed by resolved T133717: Letsencrypt all the prod things we can - planning / T194962: Create and deploy a centralized letsencrypt service / Acme-chief (though I'm not sure if that also touched CA monitoring at all)?
Asking as tasks shouldn't remain stalled for too long.

akosiaris closed this task as Declined.Nov 9 2020, 10:22 AM
In T171157#6611644, @Aklapper wrote:
In T171157#3538015, @faidon wrote:

Setting to stalled until we decide what to actually do with the internal CA, as we're considering dropping it entirely in favour of other options.

@akosiaris / @faidon: Has this situation somehow changed by resolved T133717: Letsencrypt all the prod things we can - planning / T194962: Create and deploy a centralized letsencrypt service / Acme-chief (though I'm not sure if that also touched CA monitoring at all)?

Nope. All of those all for the public facing services, the internal CA was also for some non public facing ones.

That being said, the CA that was the trigger for this task, has expired on Jul 18th, but this time around caused no issues as very few certificates were issued by it and we seem to have moved away from using it. So, per T171157#3538015 , I think we can close this as Declined.

Asking as tasks shouldn't remain stalled for too long.

Aklapper removed a subscriber: cloud-services-team.May 24 2023, 3:03 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL