Change Details

As demonstrated in T329216 it is easy to unintentionally cause too many requests to be sent to the official docker registry, resulting in failed CI jobs for many users. Proposal: * Set up registry proxies to cache access to the docker.io registry. This proxy can be equipped with credentials to allow for a higher access rate (but that may not be necessary if it is doing its job well). ** In trusted/wmcs: Use gitlab's registry proxy feature. Figure out storage management/monitoring with assistance from ServiceOps. ** In DigitalOcean: Use the proxy function built into the standard registry implementation. * Block direct access to docker.io registry from all runners.

As demonstrated in T329216 it is easy to unintentionally cause too many requests to be sent to the official docker registry, resulting in failed CI jobs for many users. Proposal: * In all places where gitlab-runners live (wmcs, trusted, releng cloud runners), set up a registry proxy to cache access to the Docker Hub registry. * The [[ https://docs.docker.com/registry/ | standard docker registry implementation ]] already has built-in support for this type of proxy caching. https://docs.docker.com/registry/configuration/#proxy * This proxy can be equipped with credentials to isolate its Docker Hub pull activity accounting. * ~~Block direct access to docker.io registry from all runners~~ This is not necessary if we ensure that the mirror uses its own set of credentials. Notes: > Docker Hub limits the number of Docker image downloads (“pulls”) based on the account type of the user pulling the image. Pull rates limits are based on individual IP address. For anonymous users, the rate limit is set to 100 pulls per 6 hours per IP address. For authenticated users, it’s 200 pulls per 6 hour period. Users with a paid Docker subscription get up to 5000 pulls per day. If you require a higher number of pulls, you can also purchase an Enhanced Service Account add-on. > ... > A pull request is defined as up to two GET requests on registry manifest URLs (/v2/*/manifests/*). > A normal image pull makes a single manifest request. > A pull request for a multi-arch image makes two manifest requests. > HEAD requests aren’t counted. xref: https://docs.docker.com/docker-hub/download-rate-limit

As demonstrated in T329216 it is easy to unintentionally cause too many requests to be sent to the official docker registry, resulting in failed CI jobs for many users. Proposal: * Set up registry proxies to cache access to the docker.io registry. This proxy can be equipped with credentials to allow for a higher access rate (but that may not be necessary if it is doing its job well)In all places where gitlab-runners live (wmcs, trusted, releng cloud runners), set up a registry proxy to cache access to the Docker Hub registry. ** In trusted/wmcs: Use gitlab's registry proxy feature. Figure out storage management/monitoring with assistance from ServiceOps. The [[ https://docs.docker.com/registry/ | standard docker registry implementation ]] already has built-in support for this type of proxy caching. https://docs.docker.com/registry/configuration/#proxy ** In DigitalOcean: Use the proxy function built into the standard registry implementation* This proxy can be equipped with credentials to isolate its Docker Hub pull activity accounting. * ~~Block direct access to docker.io registry from all runners.~~ This is not necessary if we ensure that the mirror uses its own set of credentials. Notes: > Docker Hub limits the number of Docker image downloads (“pulls”) based on the account type of the user pulling the image. Pull rates limits are based on individual IP address. For anonymous users, the rate limit is set to 100 pulls per 6 hours per IP address. For authenticated users, it’s 200 pulls per 6 hour period. Users with a paid Docker subscription get up to 5000 pulls per day. If you require a higher number of pulls, you can also purchase an Enhanced Service Account add-on. > ... > A pull request is defined as up to two GET requests on registry manifest URLs (/v2/*/manifests/*). > A normal image pull makes a single manifest request. > A pull request for a multi-arch image makes two manifest requests. > HEAD requests aren’t counted. xref: https://docs.docker.com/docker-hub/download-rate-limit