Page MenuHomePhabricator
Create Task
Maniphest T391467

gitlab ci: validate secrets settings in pipeline for tofu integration
Closed, ResolvedPublic
Actions

Description

The https://gitlab.wikimedia.org/repos/cloud/cloud-vps/networktests-tofu-provisioning repository uses a new model for setting the gitlab CI and tofu integration.

For us to get tofu plan for merge requests, we need to set the secret variables as not protected so they are available in MR branches.
Per the comment in https://phabricator.wikimedia.org/T370652#10206515:

I think that for repos/cloud you can limit the secrets exposure to MRs of branches within the repo, so it gets limited to members of the cloud team (we do so in toolforge to push images/charts to toolsbeta-harbor)

This has been implemented in a similar way.

This ticket is to verify that this is correct, before we move on with similar implementations in other tofu projects, like:

In particular, we want to verify that only "trusted" accounts can send MRs in a way that triggers gitlab CI/CD. If untrusted accounts can send arbitrary MRs that triggers the pipeline, they could submit a MR with a modified gitlab-ci.yaml file, with an 'echo $SECRET' and obtain the secrets required to perform arbitrary infrastructure changes.

Related Objects
Search...

Event Timeline

• aborrero created this task.Apr 9 2025, 12:13 PM
• aborrero triaged this task as High priority.
• aborrero added a parent task: T370652: tofu-infra: introduce additional gitlab-ci automation.
• aborrero added a parent task: T390057: bootstrap Toolforge IaC automation.
taavi added a comment.Apr 9 2025, 12:14 PM

Which credentials is that using in the first place?

• aborrero added a comment.Apr 9 2025, 12:15 PM
In T391467#10725694, @taavi wrote:

Which credentials is that using in the first place?

An user called srv-networktests @ codfw1dev, application credentials.

• aborrero added a comment.Apr 9 2025, 12:19 PM
In T391467#10725694, @taavi wrote:

Which credentials is that using in the first place?

As we plan to expand and replicate this setup in multiple different directions, what credentials are being used in particular is maybe not the important point.

If the setup is insecure, no credentials should be used this way, no matter which ones. If the setup is secure, then we mostly don't care.

• aborrero added a comment.Apr 9 2025, 1:21 PM
In T391467#10725700, @aborrero wrote:
In T391467#10725694, @taavi wrote:

Which credentials is that using in the first place?

An user called srv-networktests @ codfw1dev, application credentials.

I discovered this is only partially true:

  • ec2-style credentials for the tofu state bucket, this is using srv-networktests
  • openstack API application credentials, they are my own, as I was logged as myself when generated them via labtesthorizon.

I will refresh the openstack API application credentials to be from srv-networktests as well, soon.

• aborrero added a comment.Apr 10 2025, 8:28 AM

! In T391467#10726025, @aborrero wrote:
I will refresh the openstack API application credentials to be from srv-networktests as well, soon.

done

• aborrero moved this task from Backlog to Radar/observer on the User-aborrero board.Apr 10 2025, 8:32 AM
• aborrero added a comment.Apr 16 2025, 3:23 PM

data point: a MR from a contributor fork did not have access to the secrets in the pipeline: https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-builder/-/merge_requests/70
https://gitlab.wikimedia.org/don-vip/builds-builder/-/jobs/484613

image.png (494×1 px, 128 KB)

• aborrero added a comment.Apr 23 2025, 3:22 PM

data point: the setup was also extended to cover tofu in here: https://gitlab.wikimedia.org/repos/cloud/toolforge/tofu-provisioning/-/merge_requests/4

fnegri closed this task as Resolved.EditedMay 6 2025, 2:13 PM
fnegri claimed this task.

I found some GitLab documentation about this:

By default, pipelines from forked projects can’t access the CI/CD variables available to the parent project. If you run a merge request pipeline in the parent project for a merge request from a fork, all variables become available to the pipeline.
(from https://docs.gitlab.com/ci/variables/#cicd-variable-security)

You must be a member of the parent project with permissions to run CI/CD pipelines.
To use the UI to run a pipeline in the parent project for a merge request from a fork project:

  • In the merge request, go to the Pipelines tab.
  • Select Run pipeline. You must read and accept the warning, or the pipeline does not run.

(from https://docs.gitlab.com/ci/pipelines/merge_request_pipelines/#run-pipelines-in-the-parent-project)

My understanding from this is that there is no risk of leaking credentials unless either a) the untrusted account has write access to the repo and can create a MR in the repo and not in a fork, or b) someone with write access to the repo manually hits "run pipeline" for an untrusted fork without checking the content first.

In my view, this gives us sufficient protection against possible malicious users. I'm gonna mark this task as resolved, please reopen if you think we should do additional checks.

fnegri edited projects, added cloud-services-team (FY2024/2025-Q3-Q4); removed cloud-services-team.May 6 2025, 2:21 PM
fnegri moved this task from Backlog to Done on the cloud-services-team (FY2024/2025-Q3-Q4) board.
• aborrero mentioned this in T393686: tofu-provisioning: factorize gitlab pipeline logic.May 8 2025, 11:14 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits