Page MenuHomePhabricator

Decom cookbook should also remove keytabs and principals
Open, MediumPublic

Description

If Kerberos keytabs were issued for a host, they should also be revoked when running the decom cookbook (currently happens manually).

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 23 2020, 8:19 AM
Marostegui triaged this task as Medium priority.Oct 23 2020, 8:54 AM
Marostegui moved this task from Backlog to Acknowledged on the Operations board.

Reporting in here a chat with Moritz about steps to do. Every host that runs a keytab holds several things, let's pick analytics1028 as example:

  1. A dir in puppet private /srv/private/modules/secret/secrets/kerberos/keytabs/analytics1028.eqiad.wmnet
  2. A dir on krb1001 /srv/kerberos/keytabs/analytics1028.eqiad.wmnet
  3. Some kerberos principals registered on kadmin/kdc - hdfs/analytics1028.eqiad.wmnet@WIKIMEDIA, etc..

We could start with a read-only step, that checks all the above and reports a prompt to the user asking for inputs ("please acknowledge that you are aware of the keytabs to remove" or similar). Thoughts?

Agreed. Printing the steps is a good first step which will prevent that these files slip through. Mid-term it would be nice to have a dedicated Spicerack module for Kerberos, which wraps common kadmin commands and makes them available for cook books.

Change 642333 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] kerberos: add 'list' action to manage_principals.py

https://gerrit.wikimedia.org/r/642333

Change 642333 merged by Elukey:
[operations/puppet@production] kerberos: add 'list' action to manage_principals.py

https://gerrit.wikimedia.org/r/642333

elukey renamed this task from Decom cookbook should also remove keytabs to Decom cookbook should also remove keytabs and principals.Mon, Nov 23, 7:41 AM

Change 643000 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/cookbooks@master] sre.hosts.decommission: add support for kerberos configs

https://gerrit.wikimedia.org/r/643000

Change 643003 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] kerberos: improve manage_principals.py's parsing logic for delete

https://gerrit.wikimedia.org/r/643003

Change 643003 merged by Elukey:
[operations/puppet@production] kerberos: improve manage_principals.py's parsing logic for delete

https://gerrit.wikimedia.org/r/643003

Change 643005 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] kerberos: warn to update data.yaml on when deleting user principals

https://gerrit.wikimedia.org/r/643005

Change 643005 merged by Elukey:
[operations/puppet@production] kerberos: warn to update data.yaml only when deleting user principals

https://gerrit.wikimedia.org/r/643005

Change 643000 merged by Elukey:
[operations/cookbooks@master] sre.hosts.decommission: add support for kerberos configs

https://gerrit.wikimedia.org/r/643000

Change 643055 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] cumin: add the kerberos-kadmin alias

https://gerrit.wikimedia.org/r/643055

Change 643061 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/cookbooks@master] sre.hosts.decommission: add step to find kerberos credentials

https://gerrit.wikimedia.org/r/643061

Change 643055 merged by Elukey:
[operations/puppet@production] cumin: add the kerberos-kadmin alias

https://gerrit.wikimedia.org/r/643055