Page MenuHomePhabricator

[components-api] split source from config
Open, MediumPublicFeature

Description

Feature summary (what you would like to be able to do and where):

Currently the source of the config is specified within the config.

Being able to specify the source configuration outside of the components (today that fails validation enforcing at least 1 component exists) would reduce the risk of accidentally deploying a config with the wrong source_url, breaking the new deployment (i.e. not immediately obvious).

Use case(s) (list the steps that you performed to discover that problem, and describe the actual underlying problem which you want to solve. Do not describe only a solution):

When storing component configurations in a external source of truth, such as a git repo, it is sometimes desirable to deploy alternate versions to different projects.

As an example, deploying the component config to staging and validating (from a branch) prior to deploying to production.

Today this isn't really possible as the source_url (next up source_repo) gets flipped to what is in the external file, from what was specified during create.

Benefits (why should this be implemented?):

This would enable more flexible usage of the components service by maintainers ex. T401868, T401388

This would reduce the likelihood of broken configurations due to the 'state' of the source being stored in 2 places

Example:

Default:

toolforge components config setup<<'EOF'
source: null
EOF

Allows:

toolforge components config create<<'EOF'
components:
  pony:
    build:
      repository: https://very.special.pony
    run:
      command: i-am-a-unicorn
EOF

With source:

toolforge components config create<<'EOF'
source:
  url: https://very.special.pony
EOF

Refuses toolforge components config create, toolforge components config show causes refresh from remote (same as deploy)

This secondary structure would allow for further configuration that is not component related, for example perhaps how many builds to retain based on pattern.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

I think it's too many different places to configure, one of the goals of the tool config is to have everything in one place (say a yaml file), so you don't need to go different places to setup your tool.

Said that, I agree that the config after the source_url/source/etc. is going to be dropped, so we could make it optional, and even give back a warning (the components passed will be overridden by the config from ...). Will have to be a bit careful on how to express that though.

In that sense, we can add a 'metadata' section to gather all that info, though for now maybe keeping it at the top level is good enough (once we have more things to add to the metadata of the config we can group them to it's own structure).

I think it's too many different places to configure, one of the goals of the tool config is to have everything in one place (say a yaml file), so you don't need to go different places to setup your tool.

It can be in the same file (config about can be read as components), if that file can be 'patched' rather than 'replaced' i.e. source: xxx without components: xxx during 'setup' then replacing components: xxx without touching source: xxx during updating.

Today it has to be managed/configured in multiple places i.e. what is in the file and what is passed into the api have to match.

In that sense, we can add a 'metadata' section to gather all that info, though for now maybe keeping it at the top level is good enough (once we have more things to add to the metadata of the config we can group them to it's own structure).

A general metadata section would be nice to stuff things like the resolved ref from the repository to make viewing the 'calculated' state easy.

fgiunchedi triaged this task as Medium priority.Sep 1 2025, 2:39 PM