Page MenuHomePhabricator

Allow GitLab CI containers to connect to services
Open, Needs TriagePublic

Description

In T339352: Create MySQL container in CI for integration tests, we used the services: key in GitLab CI to define a mariadb service for CI jobs for iPoid-Service. (That also involved safe-listing this image in the operations/puppet repo; see T339352 for the relevant patches.)

However, now that I want to write integration tests which make use of the MariaDB container, I find that I get "Connection refused" errors. After installing iputils-ping, it looks like the container used for tests (nodejs16-slim) can't reach the mariadb host.

We need to be able to connect to the MariaDB container in order to have CI for integration tests (T339846: Write integration tests for data import and updates, T344802: Add tests for importing and updating data scripts called from main.sh)

Previous discussion for how one might connect to a DB from GitLab CI was in T308789#7990694, but I believe that since that conversation (June 2022) we've moved in the direction of using the services option in GitLab CI.

Event Timeline

The problem is that the build is running inside of a container created by buildkitd while the mariadb service is running in a container created by the runner. These two containers are on separate virtual networks.

The problem is that the build is running inside of a container created by buildkitd while the mariadb service is running in a container created by the runner. These two containers are on separate virtual networks.

Right... is it possible that some networking option (like this one, FF_NETWORK_PER_BUILD) would help?

It sounds like you'd want something more like the Tool-ducttape tool that QTE are planning to re-build for general use on GitLab.

@dduvall @jnuche and I discussed this a bit on IRC last week, copying the relevant lines here:

<dduvall> kostajh, jnuche: you can use gitlab ci services, but not with `.kokkuri:build-and-run-image`. the reason for this is that `.kokkuri:build-and-run-image` uses the blubber `run-variant` option to run the entrypoint within the context of buildkitd. buildkitd containers are isolated within their own CNI managed networks
17:51:59 <dduvall> to make use of services, you'd have to use a different pattern: to first build the test runner image in one job using `.kokkuri.build-image`, and then reference that built image as `image: `${<PREVIOUS_JOB_NAME>_IMAGE_REF}` in a subsequent job where you've also defined various services. using this pattern, the image built by blubber is now running as a container in the usual GitLab CI runner context
17:52:12 <dduvall> TL;DR we need to add this usage to the kokkuri docs
17:52:49 <dduvall> also, i don't think this latter usage can be supported in WMCS at the moment since we don't have a central caching registry. it's only available on DO
19:32:19 <dduvall> a good example of this pattern can be found in blubber's `.gitlab-ci.yml` https://gitlab.wikimedia.org/repos/releng/blubber/-/blob/main/.gitlab-ci.yml#L58
 here the `run-acceptance-tests` job is actually referencing two images that were built from prior jobs
 one built during the `build-frontend-for-testing` job now referenced as `${BUILD_FRONTEND_FOR_TESTING_IMAGE_REF}`, and another built during the `build-acceptance-runner` job now referenced as `${BUILD_ACCEPTANCE_RUNNER_IMAGE_REF}`. notice the variables that hold the ref value of the previously built images are named after the jobs
 dduvall> re: DO vs WMCS you're on the WMCS runners atm. you can see the hostname of the runner at the top of the console output
19:36:17 <dduvall> to ensure you get a DO runner, add `kubernetes` to the tags (ambiguous tag naming, i know)