In multiple tasks and discussions the question came up what external resources should be allowed for GitLab CI Runners (T312961, T291978, T295481). External resources is quite generic and this covers multiple areas, which are split up in the following sections. It could make sense to spin off sub-task for individual resources.
General access to public resources (egress traffic)
This is about what outgoing traffic is allowed for Runners. Should Runners be allowed to access internet resources?
Options here could be either fully unrestricted, over the webproxy or disable egress completely.
Currently Shared Runners in WMCS have mostly unrestricted access and Trusted Runners offer egress access over the webproxy.
Public package repositories
CI builds sometimes need additional packages, either for performing CI tasks or for building the artifact. So should Runners be allowed to use common package registries for CI jobs, like pip or npm? Some sources are present/mirrored in WMF infrastructure (like apt repo), some aren't.
Currently all Runners can install packages from public repositories (if available over http/https using the webproxy).
Docker images for CI purposes
Certain CI jobs use pre-build images to perform common tasks like linting, testing or code scans. Should Runners be allowed to run external images for the purpose of certain CI jobs? Please note this is not about base images (next chapter), it's only about what images can be executed during CI jobs to perform certain tasks.
Currently we restrict what images can be executed. The current list contains:
allowed_images = [ # Everything in Wikimedia registry: "docker-registry.wikimedia.org/**/*", "docker-registry.discovery.wmnet/**/*", # Distributions: "centos/*:*", "debian:*", "fedora:*", "opensuse/*:*", "ubuntu:*", # Language-specific: "python:*", "ruby:*", "rust:*", "rustlang/rust:nightly", # GitLab upstream - includes security analyzers and terraform images: "registry.gitlab.com/gitlab-org/**/*",
see config.toml.
This list is used for both Shared and Trusted Runners. There was some discussion in T312961 of adding additional security scanners which opened the discussion and this task.
Docker base images for building images
What baseimage are allow for building images for wmf/production registry? So what sources should be allowed for directly building artifacts running in production? (base in blubber or FROM field).
Other open questions regarding base images:
Is it possible to restrict this baseimages in buildkitd?
Somehow related docs: https://wikitech.wikimedia.org/wiki/Kubernetes/Images
Difference between Shared and Trusted Runners
Furthermore some of this resources may be different between the different tiers of Runners. Shared Runners could theoretically execute a wider range of images or build non-production images with a wider range of baseimages.
Currently Shared and Trusted Runners have the same access to external resources and Docker images, beside the webproxy. It should be discussed if this is reasonable for the future or if different allow-lists and policies are needed here.