As demonstrated in T329216 it is easy to unintentionally cause too many requests to be sent to the official docker registry, resulting in failed CI jobs for many users.
- In all places where gitlab-runners live (wmcs, trusted, releng cloud runners), set up a registry proxy to cache access to the Docker Hub registry.
- The standard docker registry implementation already has built-in support for this type of proxy caching. https://docs.docker.com/registry/configuration/#proxy
- This proxy can be equipped with credentials to isolate its Docker Hub pull activity accounting.
- Configure (or provide advice on how to configure) image-downloading programs to check the mirror before accessing to official registry.
Block direct access to docker.io registry from all runnersThis is not necessary if we ensure that the mirror uses its own set of credentials.
Docker Hub limits the number of Docker image downloads (“pulls”) based on the account type of the user pulling the image. Pull rates limits are based on individual IP address. For anonymous users, the rate limit is set to 100 pulls per 6 hours per IP address. For authenticated users, it’s 200 pulls per 6 hour period. Users with a paid Docker subscription get up to 5000 pulls per day. If you require a higher number of pulls, you can also purchase an Enhanced Service Account add-on.
A pull request is defined as up to two GET requests on registry manifest URLs (/v2/*/manifests/*).
A normal image pull makes a single manifest request.
A pull request for a multi-arch image makes two manifest requests.
HEAD requests aren’t counted.