Page MenuHomePhabricator

Track/alert cassandra certs expiration
Closed, ResolvedPublic

Description

followup from T108953: Cassandra inter-node encryption (TLS) is to track expirations of cassandra CA and certs, rollout involves generating new certs via cassandra-ca-manager and restart cassandra after a puppet run to deploy the updated certs

Event Timeline

fgiunchedi claimed this task.
fgiunchedi raised the priority of this task from to Medium.
fgiunchedi updated the task description. (Show Details)
fgiunchedi added a subscriber: fgiunchedi.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 7 2015, 3:55 PM
Dzahn added a subscriber: Dzahn.Dec 15 2015, 6:07 PM
Dzahn added a comment.Dec 15 2015, 6:18 PM

We can use the same method as in T116332 here with modules/nagios_common/files/check_commands/check_ssl_certfile that we can install where the cert is (like here and execute via NRPE. It runs "openssl x509 -checkend 324000 -noout -in $1 on the cert file. This is the method for services that use TLS but are not https or ldaps. For these protocols we have other plugins that actually connect to the server instead of just checking the certificate as a file.

another approach would be also to connect to the host directly and use check_ssl, possibly some changes/tweaks are needed too on the options

$ modules/nagios_common/files/check_commands/check_ssl -H localhost  -r cert.crt -p 9999  --nosni
Cannot determine hostname of peer for verification. Disabling default hostname verification for now. Please specify hostname with SSL_verifycn_name and better set SSL_verifycn_scheme too.
Label too long at /usr/share/perl5/IO/Socket/SSL.pm line 1579.

additionally, an x509 cert in PEM format can be extracted from the ca file with keytool -export -alias rootca -file cert.crt -rfc -keystore server.trust

Eevans raised the priority of this task from Medium to High.Jul 28 2016, 3:37 PM
Eevans added a project: Cassandra.
Eevans moved this task from Backlog to Next on the Cassandra board.Aug 3 2016, 3:08 PM
Eevans renamed this task from track/alert cassandra certs expiration to Track/alert cassandra certs expiration.Aug 15 2016, 7:52 PM

Change 305633 had a related patch set uploaded (by Filippo Giunchedi):
cassandra: add instance ssl monitoring

https://gerrit.wikimedia.org/r/305633

Change 305633 merged by Dzahn:
cassandra: add instance ssl monitoring

https://gerrit.wikimedia.org/r/305633

Change 305711 had a related patch set uploaded (by Dzahn):
cassandra: limit SSL cert monitoring to restbase hosts

https://gerrit.wikimedia.org/r/305711

Change 305711 merged by Dzahn:
cassandra: limit SSL cert monitoring to restbase hosts

https://gerrit.wikimedia.org/r/305711

Mentioned in SAL [2016-08-22T17:48:28Z] <godog> cassandra: replace certs for restbase-test200[123]-[ab] - T120662

Eevans moved this task from Next to In-Progress on the Cassandra board.Aug 24 2016, 4:33 PM

Change 307251 had a related patch set uploaded (by Filippo Giunchedi):
cassandra: add ssl monitoring only for ssl-enabled hosts

https://gerrit.wikimedia.org/r/307251

Change 307251 merged by Dzahn:
cassandra: add ssl monitoring only for ssl-enabled hosts

https://gerrit.wikimedia.org/r/307251

Dzahn added a comment.EditedAug 30 2016, 6:08 AM

merged and checked on neon. number of checks was 60 before and after / no-op. and yea, that was the better solution that i was looking for instead of my quick fix by hostname. thank you!

Dzahn added a comment.Aug 30 2016, 6:09 AM

guess it's resolved now?

fgiunchedi closed this task as Resolved.Aug 30 2016, 8:43 AM

yes! thanks @Dzahn, resolving

Eevans awarded a token.Sep 1 2016, 3:04 PM