The nodejs clients [[http://docs.datastax.com/en/drivers/nodejs/2.1/module-policies_loadBalancing-DCAwareRoundRobinPolicy.html|DCAwareRoundRobinPolicy]] allows client fail-over to nodes in another DC in the event no local nodes are available. Enabling this is desirable, but implicitly requires that we encrypt the client connection.
Configuring the nodejs client is fairly straightforward, but like with T108953, we will need keys and certificates generated for each node/instance, and a copy of the node/instance certificates from T108953 available in PEM format to each node/instance.
```
lang=javascript
var options = {
policies: {
loadBalancing: new cassandra.loadBalancing.DCAwareRoundRobinPolicy()
},
sslOptions: {
key: '...', // string or Buffer containing the certificate in PEM format
cert: '...', // string or Buffer containing the certificate in PEM format
ca: '...' // an array of strings of Buffers containing trusted certs in PEM format
},
};
var client = new Client(options);
```
http://docs.datastax.com/en/developer/nodejs-driver/2.2/common/drivers/reference/clientOptions.html
== Proposed rollout strategy (eqiad) ==
1. Update RESTBase config for `defaultConsistency` of `localOne`
1. Disable puppet on restbase100[1-9].eqiad nodes
1. Merge `cassandra.yaml` config change that enables client encryption
1. Reenable puppet and restart Cassandra on 1001, 1003, and 1005, applying new encryption settings from #3.
1. Rollout RESTBase configuration change to enable client encryption
1. Restart Cassandra on 1002, 1004, 1006, and 100[7-9], (applying new encryption settings from #2).
1. Restore RESTBase config for `defaultConsistency` to `localQuorum`
The idea here being that when client encryption is enabled on 1001, 1003, and 1005, existing restbase instances that would connect to these, will fail-over to the remaining nodes of the cluster when unable to connect in the clear (1, 3, and 5 represent nodes in each of racks A, B, and C). Next, RESTBase is reconfigured to use encryption and restarted, with connections failing over as needed to 1001, 1003, and 1005 when encrypted connections to 1002, 1004, 1006, and 100[7-9] fail. Finally, the client encryption settings are applied to remaining Cassandra nodes (1002, 1004, 1006, and 100[7-9]). Setting the consistency level to `localOne` ensures that queries succeed during the window when only 1 node in each rack is accessible.
== Alternative rollout strategy (eqiad) ==
1. Configure codfw nodes for client encryption
1. Update RESTBase config to set `localDc` to `codfw`
1. Rollout RESTBase configuration change to enable client encryption, (and restart RESTBase)
1. Merge `cassandra.yaml` config change that enables client encryption (and restart eqiad Cassandra nodes)
1. Restore RESTBase config for `localDc` to `eqiad`
See also: T108953