We have been running with the /v1/restricted prefix handled by a separate docker registry instance, and I'd like to do the same with the /v1/ml one since we have a dedicated Docker registry instance only for it. Main goals:
- Separate ML's requirements from the rest, so that we'll be able in the future to bump their maximum layer size limit if needed (client_max_body_size can be specified in Nginx's location contextes, so we could ideally bump the tmpfs without changing the overall restrictions).
- Test a procedure to migrate existing images to S3/apus.
I had an idea for the second point, that is not super nice and clean but it seems simple and effective:
- We pull the /ml/vllm image on a build host.
- We tag the image in a way that it can be pushed to registry2005.codfw.wmnet:$PORT (with $PORT set with the ML Registry's listening port)
- We allow buildXXXX to push to registry2005 via that port, using an iptables rule.
- We push the image, that will end up on S3/apus.
- We flip the nginx config to use the new backend.
Last but not the least, we'll need to update the Python script that generates the home dir for docker-registry.wikimedia.org, to include the new source (otherwise it won't get published and it will be confusing).