Page MenuHomePhabricator

Write generic certificate management software for use with Puppet and Self Signing CAs.
Closed, ResolvedPublic21 Story Points

Description

The casssandra-ca-manager exists to ease generation and distribution of Java keystore files to enable encryption between Cassandra clients and cluster nodes. However, this script doesn't have anything Cassandra specific in it, and could be reused to enable encryption for Kafka traffic.

puppet_ecdsacert.rb exists to help signing of certificates with our Puppet CA infrastructure, but does not include tooling for managing and declaring many certificates. create_ecdsa_cert helps a bit, but only works for generating one certificate at a time. puppet_ecdsacert.rb only generates .pem format certificate files, and both Kafka and Cassandra need Java keystores.

We should have a generic way of managing certificates that will work for both of these use cases, and hopefully future ones.

I plan to adapt Eric's cassandra-ca-manager, but make it generic and extensible, so that it works with more CAs than just self-signing ones.

Details

Related Gerrit Patches:

Event Timeline

Ottomata created this task.May 23 2017, 7:31 PM
Ottomata updated the task description. (Show Details)
Nuria edited projects, added Analytics-Kanban; removed Analytics.

Change 355782 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] [WIP] Genericize ca-manager

https://gerrit.wikimedia.org/r/355782

Ottomata claimed this task.May 26 2017, 3:58 PM
Ottomata moved this task from Next Up to In Progress on the Analytics-Kanban board.
Ottomata set the point value for this task to 8.Jun 1 2017, 8:26 PM

Change 359960 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/software/certpy@master] Initial commit of certpy

https://gerrit.wikimedia.org/r/359960

Ottomata renamed this task from Genericize ca-manager script to Write generic certificate management software for use with Puppet and Self Signing CAs..Jun 19 2017, 5:24 PM
Ottomata updated the task description. (Show Details)
Ottomata changed the point value for this task from 8 to 21.
Ottomata added a subscriber: Joe.

I've moved my work from my github account to gerrit under a new project name 'certpy'. It is still WIP and there are lots of TODOs littered throughout the code, but the basic idea is present.

I'd like to get a review now to see if I am way off from what Opsen will like. (cough cough @Joe :) )

https://gerrit.wikimedia.org/r/359960

Sorry its a lot of code, but I tried to keep it pretty clean and documented.

elukey added a subscriber: Volans.Jun 19 2017, 5:56 PM
Ottomata moved this task from Backlog to Kafka on the Analytics-Cluster board.Jul 11 2017, 3:02 PM

FYI, etherpad overview and discussion here: https://etherpad.wikimedia.org/p/certpy

Change 376592 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[cergen@master] Initial commit of cergen

https://gerrit.wikimedia.org/r/376592

Change 359960 abandoned by Ottomata:
Initial commit of cergen

Reason:
Renaming this module to cergen. Abandoning in favor of https://gerrit.wikimedia.org/r/#/c/376592/

https://gerrit.wikimedia.org/r/359960

Nuria moved this task from In Progress to Paused on the Analytics-Kanban board.Sep 14 2017, 4:19 PM

Change 355782 abandoned by Ottomata:
Genericize ca-manager

Reason:
https://gerrit.wikimedia.org/r/#/c/376592/ is better

https://gerrit.wikimedia.org/r/355782

Change 385237 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[integration/config@master] Enable tox for python cergen

https://gerrit.wikimedia.org/r/385237

Change 385237 merged by jenkins-bot:
[integration/config@master] Enable tox for python cergen

https://gerrit.wikimedia.org/r/385237

Change 376592 merged by Ottomata:
[cergen@master] Initial commit of cergen

https://gerrit.wikimedia.org/r/376592

Change 389787 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[cergen@master] Initial debian commit

https://gerrit.wikimedia.org/r/389787

Change 389788 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[cergen@master] Bump versions of cryptography and pyOpenSSL to needed versions

https://gerrit.wikimedia.org/r/389788

Change 389788 merged by Ottomata:
[cergen@master] Bump versions of cryptography and pyOpenSSL to needed versions

https://gerrit.wikimedia.org/r/389788

Change 389786 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[cergen@debian] Initial debian commit

https://gerrit.wikimedia.org/r/389786

Change 389787 abandoned by Ottomata:
Initial debian commit

Reason:
Wrong branch

https://gerrit.wikimedia.org/r/389787

Change 391134 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] [WIP] Add cergen module

https://gerrit.wikimedia.org/r/391134

Change 394106 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[cergen@master] Generate truststore jks file when generating Certificate files

https://gerrit.wikimedia.org/r/394106

Change 394144 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Puppetize SSL for Kafka broker

https://gerrit.wikimedia.org/r/394144

Change 394144 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Puppetize SSL for Kafka broker

https://gerrit.wikimedia.org/r/394144

Change 394106 merged by Ottomata:
[cergen@master] Generate truststore jks file when generating Certificate files

https://gerrit.wikimedia.org/r/394106

Change 394310 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Install cergen on Puppet CA host

https://gerrit.wikimedia.org/r/394310

Change 389786 merged by Ottomata:
[cergen@debian] Initial debian commit

https://gerrit.wikimedia.org/r/389786

Change 394310 merged by Ottomata:
[operations/puppet@production] Install cergen on Puppet CA host

https://gerrit.wikimedia.org/r/394310

Change 394314 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Install some cergen python3 deps from jessie-backports

https://gerrit.wikimedia.org/r/394314

Change 394314 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Install some cergen python3 deps from jessie-backports

https://gerrit.wikimedia.org/r/394314

Change 394314 merged by Ottomata:
[operations/puppet@production] Install some cergen python3 deps from jessie-backports

https://gerrit.wikimedia.org/r/394314

Change 394325 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Need to specify more dependencies for cergen, create cergen module

https://gerrit.wikimedia.org/r/394325

Change 394325 merged by Ottomata:
[operations/puppet@production] Need to specify more dependencies for cergen, create cergen module

https://gerrit.wikimedia.org/r/394325

Change 394327 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] require python3-pkg-resources for cergen in jessie

https://gerrit.wikimedia.org/r/394327

Change 394327 merged by Ottomata:
[operations/puppet@production] require python3-pkg-resources for cergen in jessie

https://gerrit.wikimedia.org/r/394327

Change 394328 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Use install_options instead of manually specifying cergen deps

https://gerrit.wikimedia.org/r/394328

Change 394328 merged by Ottomata:
[operations/puppet@production] Use install_options instead of manually specifying cergen deps

https://gerrit.wikimedia.org/r/394328

Change 394144 merged by Ottomata:
[operations/puppet@production] Puppetize SSL for Kafka broker

https://gerrit.wikimedia.org/r/394144

Ottomata moved this task from In Code Review to Done on the Analytics-Kanban board.Dec 8 2017, 3:20 PM
Nuria closed this task as Resolved.Dec 12 2017, 5:11 PM

Change 391134 abandoned by Ottomata:
[WIP] Add cergen module

https://gerrit.wikimedia.org/r/391134