Page MenuHomePhabricator

CI on gitlab for eBPF / networking heavy projects
Open, MediumPublic

Description

We have some projects on gitlab (liberica, tcp-mss-clamper, ipip-multiqueue-optiimizer) that have some unique CI/testing requirements:

  • Load eBPF programs on kernel
  • Be able to test traffic from a LB acting kernel to realservers

This could potentially trigger severe networking issues on gitlab runners. As mentioned by @BCornwall it looks like we could leverage https://docs.gitlab.com/runner/executors/custom.html and spawn VMs using libvirt (https://docs.gitlab.com/runner/executors/custom_examples/libvirt.html)

Details

TitleReferenceAuthorSource BranchDest Branch
clamper: Provide integration tests using qemurepos/sre/tcp-mss-clamper!11vgutierrezT353279-qemumain
Customize query in GitLab

Event Timeline

Vgutierrez triaged this task as Medium priority.Dec 12 2023, 4:57 PM

I think this is a case where it would make sense to bring your own runner to GitLab—that is, this is a use-case that would be likely to disrupt other users, so a shared runner would probably be unworkable (I believe).

But a specialized runner where you had the permissions you needed would likely work. You can add a runner to your specific repo in GitLab pretty easily.

Can I ask where these projects are tested currently?

we currently perform manual tests on developer machines (far from optimal). So if we can spawn our own runner we could run docker containers in privileged mode there? It could be easier and faster than spawning VMs per CI execution

@thcipriani being able to run privileged containers seems to be enough, at least for basic eBPF tests (not sure about IPVS setups yet), https://gitlab.wikimedia.org/repos/sre/tcp-mss-clamper/-/merge_requests/10 leverages docker-compose to spawn two containers and validate that tcp-mss-clamper (small eBPF program that performs TCP MSS clamping) is working as expected

Docker is definitely not a valid option here since we need to test against several kernels (at least 5.10 and 6.1)

https://gitlab.wikimedia.org/repos/sre/tcp-mss-clamper/-/merge_requests/11 performs integration tests using qemu to spawn a bullseye and a bookworm kernel and then perform basic eBPF tasks, this should be enough for now.

@thcipriani I'm guessing that running qemu (version provided by debian) shouldn't be a big deal on gitlab runners?