Page MenuHomePhabricator

Jobs killed on GitLab DO runners
Closed, ResolvedPublic

Description

Some of SRE Traffic's jobs running on GitLab DO runners were killed, see https://gitlab.wikimedia.org/repos/sre/dnsdist/-/jobs/178353 for an example. Can these jobs be assigned to a DO runner with more resources available?

There is a somewhat related task: T353279: CI on gitlab for eBPF / networking heavy projects, the two combined may warrant a separate runner for Traffic in the long run.

Event Timeline

Try adding

tags: [memory-optimized]

to the failing job.

@dancy You're a wizard. Thanks so much for this, it's now happily building. :)

I added some docs to https://wikitech.wikimedia.org/wiki/GitLab/Gitlab_Runner#GitLab_Runner_types

BCornwall claimed this task.

Hm, I'm still running into this every now and then. @dancy, Is there a possibility of raising the amount of memory even more?

Looking at https://gitlab.wikimedia.org/repos/sre/dnsdist/-/blame/bookworm-wikimedia/.gitlab-ci.yml#L14 it seems you identified the problem with running multiple build processes simultaneously using too much memory. Did this fully resolve the problem or do you still need more memory?

Indeed, this was my mistake. Sorry for that! Thanks for your help, @dancy