Page MenuHomePhabricator

Provide a means to introduce "skew" for scheduled jobs to avoid thundering herd problems
Closed, ResolvedPublicFeature

Description

A known problem with both the grid engine cron and the new jobs framework is thundering herd problems for the execution environment schedulers at popular times (00:00, *:00, *:30, etc).

Because TJF is acting as an intermediary between the user and the Kubernetes backend we have an opportunity to introduce support for a system generated skew or offset for users attempting to avoid the thundering herd. As an intermediary, the skew that is introduced cannot easily be a dynamic response of the system. We do however have an opportunity to translate the schedule provided by the user into a slightly different set of fixed values when passing data to the Kubernetes API.

Some ideas:

Event Timeline

I like the idea of translating @hourly @daily etc behind the scenes. I'll put up a patch soon to do that unless anyone feels strongly against it.

I like the idea of translating @hourly @daily etc behind the scenes. I'll put up a patch soon to do that unless anyone feels strongly against it.

+1

Another (not mutually exclusive) idea is to introduce some kind of validation to prevent more than N jobs in the same time frame, with N being something like 2*(number of worker nodes).
This can be done at a later time, in other ticket, if we find it useful.

Change 927586 had a related patch set uploaded (by Majavah; author: Majavah):

[cloud/toolforge/jobs-framework-api@main] Parse cron expressions server-side

https://gerrit.wikimedia.org/r/927586

Change 927587 had a related patch set uploaded (by Majavah; author: Majavah):

[cloud/toolforge/jobs-framework-api@main] cron: Randomly skew cron at-macros

https://gerrit.wikimedia.org/r/927587

Change 927586 merged by jenkins-bot:

[cloud/toolforge/jobs-framework-api@main] Parse cron expressions server-side

https://gerrit.wikimedia.org/r/927586

Change 927587 merged by jenkins-bot:

[cloud/toolforge/jobs-framework-api@main] cron: Randomly skew cron at-macros

https://gerrit.wikimedia.org/r/927587

Deployed to toolsbeta:

toolsbeta.test3@toolsbeta-sgebastion-05:~$ toolforge-jobs run test-scheduler --command hostname --image bullseye --emails none --schedule "@hourly"
toolsbeta.test3@toolsbeta-sgebastion-05:~$ kubectl get cronjob
NAME             SCHEDULE     SUSPEND   ACTIVE   LAST SCHEDULE   AGE
test-scheduler   59 * * * *   False     0        <none>          7s
toolsbeta.test3@toolsbeta-sgebastion-05:~$ toolforge-jobs list
Job name:       Job type:          Status:
--------------  -----------------  --------------------------
test-scheduler  schedule: @hourly  Waiting for scheduled time