Change Details

This is a task to brainstorm ideas on how to evolve the current secrets management (plain text files on NFS) to a solution that will allow us more flexibility to change the implementation without the users needing to change any code, with that, then we can move it away from plain NFS and onto somewhere else. Some starting ideas, focusing on the user story side, without going too deep into how to implement it: === Toolforge secrets === Create a toolforge cli for secrets (`toolforge secrets`), that handles the secret management (probably as a cli-api combo) As a first implementation, it could use k8s secrets on the tool namespace as backend. That could read stdin, to allow things like "<mysecret.file toolforge secrets create --name='awesome_secret' --type=file --data=-" that creates the secret from an existing file. There could be two types of secrets (following k8s secrets), files and env vars. Common fields: * name * data File-only fields: * mountpath?: we could use a generated one instead, so it would not be needed. I would push towards having an associated env var, like SECRET_<name>_PATH, and users having to read the path from there in any case, that allows us (and them!) to not rely on any hardcodded path. As user stories, that would be something like: * as a tool, I want to be able to create an environment variable with secret data, and use it on my tool * as a tool, I want to be able to create a file with secret data, and use it on my tool Will move to a wiki once there's a clearer idea of the options

This is a task to brainstorm ideas on how to evolve the current secrets management (plain text files on NFS) to a solution that will allow us more flexibility to change the implementation without the users needing to change any code, with that, then we can move it away from plain NFS and onto somewhere else. Some starting ideas, focusing on the user story side, without going too deep into how to implement it: === Toolforge secrets, files and env vars === Create a toolforge cli for secrets (`toolforge secret`), that handles the secret management (probably as a cli-api combo) As a first implementation, it could use k8s secrets on the tool namespace as backend. That could read stdin, to allow things like "<mysecret.file toolforge secrets create --name='awesome_secret' --type=file --data=-" that creates the secret from an existing file. There could be two types of secrets (following k8s secrets), files and env vars. Common fields: * name * value File-only fields: * mountpath: we could use a generated one instead, so it would not be needed. I would push towards having an associated env var, like SECRET_<name>_PATH, and users having to read the path from there in any case, that allows us (and them!) to not rely on any hardcodded path. As user stories, that would be something like: * as a tool, I want to be able to create an environment variable with secret data, and use it on my tool * as a tool, I want to be able to create a file with secret data, and use it on my tool === Toolforge secrets, only env vars === Create a toolforge cli for secrets (`toolforge secret`), that handles the secret management. Create an alias `toolforge envvar` (or the other way around, not really relevant which one is an alias to which). This version will only support environment variables (following https://12factor.net). As a first implementation, it could use k8s secrets on the tool namespace as backend. Fields: * name * value As user stories, that would be something like: * as a tool, I want to be able to create an environment variable with secret data, and use it on my tool NOTE: If at any point we change the implementation for `envvars` and `secret`, we can just swap the alias with the new implementation. == Note == I'll move to a decision task at the end of the week.

This is a task to brainstorm ideas on how to evolve the current secrets management (plain text files on NFS) to a solution that will allow us more flexibility to change the implementation without the users needing to change any code, with that, then we can move it away from plain NFS and onto somewhere else. Some starting ideas, focusing on the user story side, without going too deep into how to implement it: === Toolforge secrets, files and env vars === Create a toolforge cli for secrets (`toolforge secrets`), that handles the secret management (probably as a cli-api combo) As a first implementation, it could use k8s secrets on the tool namespace as backend. That could read stdin, to allow things like "<mysecret.file toolforge secrets create --name='awesome_secret' --type=file --data=-" that creates the secret from an existing file. There could be two types of secrets (following k8s secrets), files and env vars. Common fields: * name * data* value File-only fields: * mountpath?: we could use a generated one instead, so it would not be needed. I would push towards having an associated env var, like SECRET_<name>_PATH, and users having to read the path from there in any case, that allows us (and them!) to not rely on any hardcodded path. As user stories, that would be something like: * as a tool, I want to be able to create an environment variable with secret data, and use it on my tool * as a tool, I want to be able to create a file with secret data, and use it on my tool Will move to a wiki once there's a clearer idea of the options=== Toolforge secrets, only env vars === Create a toolforge cli for secrets (`toolforge secret`), that handles the secret management. Create an alias `toolforge envvar` (or the other way around, not really relevant which one is an alias to which). This version will only support environment variables (following https://12factor.net). As a first implementation, it could use k8s secrets on the tool namespace as backend. Fields: * name * value As user stories, that would be something like: * as a tool, I want to be able to create an environment variable with secret data, and use it on my tool NOTE: If at any point we change the implementation for `envvars` and `secret`, we can just swap the alias with the new implementation. == Note == I'll move to a decision task at the end of the week.