Page MenuHomePhabricator

Dockerhub rate limiting a potential problem for dev environment development
Closed, ResolvedPublic


The docker development environment in mw cli relies on several images from dockerhub: adminer, defreitas/dns-proxy-server, jwilder/nginx-proxy, graphiteapp/graphite-statsd, maraidb, mysql, postgres, phpmyadmin, redis

Pulling these images in CI (currently run in github) every time a patch is uploaded could cause the rate limits to be hit and slow down the development process.

We need to figure out how to avoid the rate limits, whether it's hosting our own images or figuring out how to not always pull an image if it can be stored in a local docker registry for the next CI run.


TitleReferenceAuthorSource BranchDest Branch
.gitlab-ci.yml: Don't specify docker mirror hererepos/releng/cli!17addshoregitlab-ci-no-specified-mirrormain
Customize query in GitLab

Event Timeline

For now I have mainly got around this by adding the google cloud public mirror for docker hub that has no limits

I also wrote a job that will check the quota for the runner that mwcli is currently using

Addshore claimed this task.

We might end up encountering this again at some point, but for now everything is OK!

Addshore reopened this task as Open.EditedSep 9 2021, 3:54 PM

Of course moments after closing this it happened

toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading:

Today was quite a heavy day for development etc..

I'm going to experiment with a docker registry mirror running on the CI runner

Mainly from

Create a mirror (using docker):

sudo docker run -d -p 6000:5000 \
    --restart always \
    --name registry registry:2

Get the IP address:

hostname --ip-address

Add the mirror:

sudo service docker stop
sudo echo '{"registry-mirrors": [""]}' > /etc/docker/daemon.json
sudo service docker restart

I have "pounded" CI today after setting this up and had no issues.

And even after "all of the CI" most of the limit remains

ratelimit-limit: 100;w=21600
ratelimit-remaining: 99;w=21600