Problem

Currently we have a bunch of configuration files under /etc or /etc/toolforge, each with some information for some of the clients/tools that toolforge has, but with no specific pattern to it.

Constraints and risks

• With the addition of APIs and clients using those APIs, we have new tools needing some configuration (endpoints, auth, etc.), not having a clear place and way to put it make it confusing for new developments and debugging.
• Without a pattern, it might be harder to reuse code between tools.

Decision record

In progress

https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/EnhancementProposals/Decision_record_T336376_Decision_request_-_Find_a_standard_place_for_all_the_toolforge-related_config_files

Options

Option 1

I propose having the following structure:

/etc/toolforge <directory>
|- <client1_name>.yaml
|- <client2_name>.yaml
\- common.yaml

where client1_name is the name of the cli as installed (toolforge-webservice, toolforge-build, toolforge-jobs, ...) or the generic 'toolforge-cli' one for the top level wrapper.

The config should be read hierarchically, from lower priority to more priority:

• common.yaml
• toolforge-cli.yaml <- not sure this one is needed though, I'd remove it until it's clear it's needed
• <client_name>.yaml

And the config option put at the highest priority level that it can (if two clients need it -> common.yaml).

That can be paired with introducing in the toolforge_weld library a function to load your config, in that way and reused between client.

Pros:

• Clear configuration places and hierarchy
• Common way to load the config
• Changes in one cli config are hard to affect other clis

Cons:

• A bit of work refactoring the current code in puppet and clients
• A bunch of files to look into, but at least all together

Option 2

Having one single config file /etc/toolforge.yaml with different subsections:

common:
    api_gateway_url: "https://api.svc.tools.eqiad1.wikimedia.cloud:30003/jobs/api/v1"
builds:
    builds_endpoint: "builds/v1"
    builder_image: tools-harbor.wmcloud.org/toolforge/heroku-builder-classic:22
jobs:
    api_url: https://api.svc.tools.eqiad1.wikimedia.cloud:30003/jobs/api/v1
    kubeconfig: "~/.kube/config"
...

Paired as before for a common function to load the config.

Pros:

• Clear configuration places and hierarchy
• Common way to load the config
• One single file to look for things

Cons:

• Easier to mess up the other clients configs with a bad patch (ex. removing what you shouldn't, etc.)
• A bit of work refactoring the current code in puppet and clients

Option N

Add your option here!