In a recent Slack conversation with the releng team, we discussed how to get rid of a custom gitlab runner that was put in place on T321736. This custom runner allows us to run 'docker besides docker', with the final goal of building and running a Dockerfile that generates an production artifact.
Instead of running 'docker besides docker', we live coded a proof of concept on using buildkit to build and extract the artifact without the need of 'docker besides docker'.
Here are the draft MR with the proof of concept:
https://gitlab.wikimedia.org/repos/data-engineering/workflow_utils/-/merge_requests/36
https://gitlab.wikimedia.org/repos/data-engineering/conda-analytics/-/merge_requests/17
In this task we want to:
- Generate a custom gitlab runner image that exposes buildkit with a more user friendly interface for building and extracting an artifact.
- Test that such mechanism can replace T321736.