Change Details

In recent discussions (T351792, T356418) the need of GitLab Runners with enabled Dockerfile frontend was discussed. Currently Trusted Runners support [blubber](https://wikitech.wikimedia.org/wiki/Blubber) only. We agreed that Dockerfile frontend should not be generally available in production. For production images builds based on [blubber](https://wikitech.wikimedia.org/wiki/Blubber) provide proven security settings which are hard to replicate with raw Dockerfiles. So most images should be build with blubber. However we identified a few use-cases which need Dockerfile support. For example the build of [base-images](https://wikitech.wikimedia.org/wiki/Kubernetes/Images#Base_images) and [production-images](https://wikitech.wikimedia.org/wiki/Kubernetes/Images#Production_images) (currently handled by [docker-pkg](https://doc.wikimedia.org/docker-pkg/)). Also some images used in the CI stack (like [integration/config](https://gerrit.wikimedia.org/r/plugins/gitiles/integration/config/) or [buildkit](https://gitlab.wikimedia.org/repos/releng/buildkit)) are based on a Dockerfile. So for this uses cases and to unblock firsts tests with docker-pkg on GitLab we decided to convert one of the Trusted Runners to a special purpose Trusted Dockerfile Runner. We agreed the Runner should be available to `repos/releng` and `repos/sre` GitLab group. This clashes a bit with the current policy of Trusted Runners because the access has to be requested explicitly on a per-project basis currently. So making one Trusted Runner available to all RelEng and SRE projects would change the policy a bit. Rough todos: [x] self-build `docker.io/docker/dockerfile-upstream` with the new Dockerfile Runner: `docker-registry.discovery.wmnet/repos/releng/buildkit/dockerfile-frontend:experiment` [] decide if Trusted Dockerfile Runner should be available to all `repos/releng` and `repos/sre` or explicitly requested (similar to current Trusted Runners) [x] create a dedicated Trusted Runner type and get the token in [repos/releng/gitlab-settings/runner-config.json](https://gitlab.wikimedia.org/repos/releng/gitlab-settings/-/blob/main/runner-config.json?ref_type=heads) [] adapt automation in https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/ to make sure multiple classes of Trusted Runners can be managed [x] unregister one of the Trusted Runners [x] update `profile::gitlab::runner::token` with the new token for one of the Trusted Runners in private hiera [] add Dockerfile frontend to `profile::gitlab::runner::buildkitd_allowed_frontends` and `profile::gitlab::runner::buildkitd_allowed_gateway_sources` [] Run puppet on the new Trusted Runner [] add https://gitlab.wikimedia.org/repos/releng/buildkit as a first project to the Trusted Dockerfile Runner and test [] add a Dockerfile Runner to the test environment [] update [docs](https://wikitech.wikimedia.org/wiki/GitLab) For other, non-production use cases (like T351792) we could also start a discussion of enabling Dockerfile frontend on the Cloud/WMCS Runners in a dedicated Task. But this would require a dedicated Docker registry and additional work and maintenance.

In recent discussions (T351792, T356418) the need of GitLab Runners with enabled Dockerfile frontend was discussed. Currently Trusted Runners support [blubber](https://wikitech.wikimedia.org/wiki/Blubber) only. We agreed that Dockerfile frontend should not be generally available in production. For production images builds based on [blubber](https://wikitech.wikimedia.org/wiki/Blubber) provide proven security settings which are hard to replicate with raw Dockerfiles. So most images should be build with blubber. However we identified a few use-cases which need Dockerfile support. For example the build of [base-images](https://wikitech.wikimedia.org/wiki/Kubernetes/Images#Base_images) and [production-images](https://wikitech.wikimedia.org/wiki/Kubernetes/Images#Production_images) (currently handled by [docker-pkg](https://doc.wikimedia.org/docker-pkg/)). Also some images used in the CI stack (like [integration/config](https://gerrit.wikimedia.org/r/plugins/gitiles/integration/config/) or [buildkit](https://gitlab.wikimedia.org/repos/releng/buildkit)) are based on a Dockerfile. So for this uses cases and to unblock firsts tests with docker-pkg on GitLab we decided to convert one of the Trusted Runners to a special purpose Trusted Dockerfile Runner. We agreed the Runner should be available to `repos/releng` and `repos/sre` GitLab group. This clashes a bit with the current policy of Trusted Runners because the access has to be requested explicitly on a per-project basis currently. So making one Trusted Runner available to all RelEng and SRE projects would change the policy a bit. Rough todos: [x] self-build `docker.io/docker/dockerfile-upstream` with the new Dockerfile Runner: `docker-registry.discovery.wmnet/repos/releng/buildkit/dockerfile-frontend:experiment` [x] decide if Trusted Dockerfile Runner should be available to all `repos/releng` and `repos/sre` or explicitly requested (similar to current Trusted Runners) [x] create a dedicated Trusted Runner type and get the token in [repos/releng/gitlab-settings/runner-config.json](https://gitlab.wikimedia.org/repos/releng/gitlab-settings/-/blob/main/runner-config.json?ref_type=heads) [] adapt automation in https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/ to make sure multiple classes of Trusted Runners can be managed [x] unregister one of the Trusted Runners [x] update `profile::gitlab::runner::token` with the new token for one of the Trusted Runners in private hiera [] add Dockerfile frontend to `profile::gitlab::runner::buildkitd_allowed_frontends` and `profile::gitlab::runner::buildkitd_allowed_gateway_sources` [] Run puppet on the new Trusted Runner [] add https://gitlab.wikimedia.org/repos/releng/buildkit as a first project to the Trusted Dockerfile Runner and test [] add a Dockerfile Runner to the test environment [] update [docs](https://wikitech.wikimedia.org/wiki/GitLab) For other, non-production use cases (like T351792) we could also start a discussion of enabling Dockerfile frontend on the Cloud/WMCS Runners in a dedicated Task. But this would require a dedicated Docker registry and additional work and maintenance.

In recent discussions (T351792, T356418) the need of GitLab Runners with enabled Dockerfile frontend was discussed. Currently Trusted Runners support [blubber](https://wikitech.wikimedia.org/wiki/Blubber) only. We agreed that Dockerfile frontend should not be generally available in production. For production images builds based on [blubber](https://wikitech.wikimedia.org/wiki/Blubber) provide proven security settings which are hard to replicate with raw Dockerfiles. So most images should be build with blubber. However we identified a few use-cases which need Dockerfile support. For example the build of [base-images](https://wikitech.wikimedia.org/wiki/Kubernetes/Images#Base_images) and [production-images](https://wikitech.wikimedia.org/wiki/Kubernetes/Images#Production_images) (currently handled by [docker-pkg](https://doc.wikimedia.org/docker-pkg/)). Also some images used in the CI stack (like [integration/config](https://gerrit.wikimedia.org/r/plugins/gitiles/integration/config/) or [buildkit](https://gitlab.wikimedia.org/repos/releng/buildkit)) are based on a Dockerfile. So for this uses cases and to unblock firsts tests with docker-pkg on GitLab we decided to convert one of the Trusted Runners to a special purpose Trusted Dockerfile Runner. We agreed the Runner should be available to `repos/releng` and `repos/sre` GitLab group. This clashes a bit with the current policy of Trusted Runners because the access has to be requested explicitly on a per-project basis currently. So making one Trusted Runner available to all RelEng and SRE projects would change the policy a bit. Rough todos: [x] self-build `docker.io/docker/dockerfile-upstream` with the new Dockerfile Runner: `docker-registry.discovery.wmnet/repos/releng/buildkit/dockerfile-frontend:experiment` [x] decide if Trusted Dockerfile Runner should be available to all `repos/releng` and `repos/sre` or explicitly requested (similar to current Trusted Runners) [x] create a dedicated Trusted Runner type and get the token in [repos/releng/gitlab-settings/runner-config.json](https://gitlab.wikimedia.org/repos/releng/gitlab-settings/-/blob/main/runner-config.json?ref_type=heads) [] adapt automation in https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/ to make sure multiple classes of Trusted Runners can be managed [x] unregister one of the Trusted Runners [x] update `profile::gitlab::runner::token` with the new token for one of the Trusted Runners in private hiera [] add Dockerfile frontend to `profile::gitlab::runner::buildkitd_allowed_frontends` and `profile::gitlab::runner::buildkitd_allowed_gateway_sources` [] Run puppet on the new Trusted Runner [] add https://gitlab.wikimedia.org/repos/releng/buildkit as a first project to the Trusted Dockerfile Runner and test [] add a Dockerfile Runner to the test environment [] update [docs](https://wikitech.wikimedia.org/wiki/GitLab) For other, non-production use cases (like T351792) we could also start a discussion of enabling Dockerfile frontend on the Cloud/WMCS Runners in a dedicated Task. But this would require a dedicated Docker registry and additional work and maintenance.