Page MenuHomePhabricator

[tbs.maintaindbusers] Update maintain-dbusers to create an envvar with replica.my.cnf to be used by buildpack images
Closed, ResolvedPublic

Description

Now that we have a secrets management service (envvars service), we can make maintain-dbusers put in an envar with the credentials for the database access, that will allow a lot of tools to not need NFS at all and unblock the first NFS-less webservices.

We ended up creating 4 environment variables:

  • TOOL_DATABASE_USER/TOOL_DATABASE_PASSWORD -> user/pass for toolsdb
  • TOOL_REPLICA_USER/TOOL_REPICA_PASSWORD -> user/pass for the replicas (that currently is the same)

Event Timeline

One option I just thought of is that the deploy mechanism (or even maintain-kubeusers) could copy the contents into a secret (which is encrypted at rest in our cluster) and then we could mount it into the pod like a file.

Having maintain-kubeusers do it seems nicer as it means the deploy mechanism doesn't have to have access to any secret stuff (I haven't fully reasoned about the security model of it yet though). Is there any overhead/risk to having maintain-kubeusers create all the secrets ahead of time?

There shouldn't be. The secrets are scoped to the namespaces and the only people who can see into them are people who can do so anyway via NFS. They are encrypted in etcd.

The ideal might actually be that the service that creates the credentials in the first place ("maintain-dbusers"...very creative naming conventions are hard) on NFS also creates the secrets in my mind because then they could both be rotated the same way. To do *that*, we'd have to give it a k8s certificate, but that's not hard. We have a script for it as long as the need for the cert and its rotation is added to the docs.

maintain-dbusers needs work this quarter anyway, so it might be a good time to consider adding a new function to create secrets in k8s (probably based on the webservice methods for interacting with it to avoid pip libraries on physical servers).

The ideal might actually be that the service that creates the credentials in the first place ("maintain-dbusers"...very creative naming conventions are hard) on NFS also creates the secrets in my mind because then they could both be rotated the same way. To do *that*, we'd have to give it a k8s certificate, but that's not hard. We have a script for it as long as the need for the cert and its rotation is added to the docs.

Sounds good to me.

maintain-dbusers needs work this quarter anyway, so it might be a good time to consider adding a new function to create secrets in k8s (probably based on the webservice methods for interacting with it to avoid pip libraries on physical servers).

Would it make sense to do this before or after that other work?

Would it make sense to do this before or after that other work?

Maybe during? I'll check how bad it is :-D
It's tricky to test because the maintain-dbusers script is a PITA to test. The function to create the secrets is probably not going to be too bad to test.

Legoktm renamed this task from Figure out how to provide replica.my.cnf to buildpack images to Update maintain-dbusers to create a k8s secret with replica.my.cnf to be used by buildpack images.Oct 20 2020, 6:57 PM
Legoktm updated the task description. (Show Details)
Andrew triaged this task as Medium priority.Jan 12 2021, 5:10 PM
Andrew moved this task from Soon! to Inbox on the cloud-services-team (Kanban) board.
dcaro renamed this task from Update maintain-dbusers to create a k8s secret with replica.my.cnf to be used by buildpack images to [tbs.maintaindbusers] Update maintain-dbusers to create a k8s secret with replica.my.cnf to be used by buildpack images.Aug 26 2022, 8:57 AM
dcaro renamed this task from [tbs.maintaindbusers] Update maintain-dbusers to create a k8s secret with replica.my.cnf to be used by buildpack images to [tbs.maintaindbusers] Update maintain-dbusers to create an envvar with replica.my.cnf to be used by buildpack images.Jun 20 2023, 12:18 PM
dcaro updated the task description. (Show Details)

Change 933973 had a related patch set uploaded (by David Caro; author: David Caro):

[operations/puppet@production] replica_cnf_api: refactor to use multiple backends

https://gerrit.wikimedia.org/r/933973

dcaro changed the task status from Open to In Progress.Jul 3 2023, 4:46 PM
dcaro claimed this task.
dcaro moved this task from Next Up to In Progress on the Toolforge Build Service (Iteration 16) board.

Change 936232 had a related patch set uploaded (by David Caro; author: David Caro):

[operations/puppet@production] WIP: replica_cnf_api: add envvars backend

https://gerrit.wikimedia.org/r/936232

Change 933973 merged by David Caro:

[operations/puppet@production] replica_cnf_api: refactor to use multiple backends

https://gerrit.wikimedia.org/r/933973

Change 936232 merged by David Caro:

[operations/puppet@production] replica_cnf_api: add envvars backend

https://gerrit.wikimedia.org/r/936232

dcaro updated the task description. (Show Details)
dcaro moved this task from In Review to Done on the Toolforge Build Service (Iteration 19) board.

Up and running!