Implement a secure configuration for the RESTBase Cassandra cluster; This is a tracking ticket to organize the various existing issues (some in-progress), and any additional issues that may be needed.
|Resolved||Eevans||T94329 secure Cassandra/RESTBase cluster|
|Resolved||Eevans||T92590 use non-default credentials when authenticating to Cassandra|
|Open||Eevans||T92471 enable authenticated access to Cassandra JMX|
|Resolved||Eevans||T92560 move cassandra submodule into puppet repo|
|Resolved||fgiunchedi||T94741 Update Cassandra package in Jessie to 2.1.4|
|Resolved||Dzahn||T92680 iptables firewall to limit access to Cassandra services|
|Resolved||Dzahn||T94330 disable Thrift RPC interface on Cassandra/RESTBase cluster|
|Resolved||Eevans||T108953 Cassandra inter-node encryption (TLS)|
|Resolved||RobH||T111382 codfw 3x spares for cassandra encryption testing|
|Resolved||Eevans||T113622 replace default Cassandra superuser|
@GWicke, the only remaining blocker is T92471: enable authenticated access to Cassandra JMX, which has languished for long enough that I think it's fair to ask whether or not it's important enough to worry about. So here goes...
There is no end to the damage you could do to the cluster using nodetool, (and the open RMI port it requires will allow remote execution of arbitrary code as user cassandra). However, firewall rules currently limit access to only the other machines in the Cassandra cluster. In other words, unauthenticated access is provided to anyone with access to the nodes of the cluster (provided the firewall remains intact), which is probably (roughly) what we want.
The question then: Is what we have good enough, or do we need the defense-in-depth that authenticated access would provide?
From a cost / benefit perspective, improving on the Firewall with a separate VLan might be more attractive than setting up JMX authentication. The focus on Kubernetes this quarter could improve our ability to set up VLans, which could bring this within reach in the foreseeable future. See T121240 for the general network segmentation discussion.
VLANs and firewalling are two distinct things, you mean adding hardware firewalls in the communications between the VLANs? Either way, it's a long shot from our current model.
The focus on Kubernetes this quarter could improve our ability to set up VLans,
I don't think this is accurate at all. Kubernetes will most probably use Calico as a networking solution, and that doesn't mean creating network segregation at all (which is what I think you want).
In terms of the concrete JMX protection question, the main proposals are:
- Set up JMX authentication, with credentials stored accessible to shell (but not service) users on Cassandra nodes.
- Add a firewall rule that only allows outgoing connections to JMX ports within the Cassandra cluster from a non-service group.
In terms of threats, differences seem to be:
- Code execution on Cassandra nodes.
- JMX authentication: User privileges need to be elevated to a non-service user / group to provide access to the credentials.
- Firewall rule: User privileges need to be elevated to a non-service user / group to be able to connect to JMX port.
- Other nodes circumventing the IP-based firewall by configuring a Cassandra cluster node IP.
- JMX authentication: Passwords offer second line of defense.
- Firewall rule: No defense.
So JMX auth would add a second line of defense for firewall circumvention, at the cost of some added complexity in tool wrappers and credential distribution.