Page MenuHomePhabricator

toolforge-jobs: load jobs from a file
Closed, ResolvedPublic

Description

Is there a way to dump/export the toolforge-jobs config so it can be committed in Git? And then a way to import that config back. I tend to commit crontabs for documentation and transparency purposes.

Have been thinking on this for a while. I think we can use yaml, something like:

---
- name: myjob1
  command: ./mycommand.sh
  image: tf-buster-std
- name: myscheduledjob
  command: ./myothercommand.sh
  image: tf-buster-whatever
  schedule: * * * * *
- name: mycontinuousjob
  command: ./myneverendingcommand.sh
  image: tf-buster-whatever
  continuous: true

Other alternative I have been considering is having the CLI be kind of an interpreter, so you could basically chain commands in a file:

tool@toolforge:~$ cat file.txt
flush
run myjob1 --image tb-buster-std --command ./mycommand.sh
run myjob2 --image whatever --command whatever --schedule "*/1 * * * *"
run myjob3 --image whatever --command whatever --continuous
run myjob4 --image whatever --command whatever --wait
delete myjob3
tool@toolforge:~$ toolforge-jobs load file.txt

This last option seems a bit over-engineered at this point, but could enable some interesting use cases. The same can be achieved with a simple shell script anyway, so I'm basically discarding that.

Event Timeline

I think a YAML file would be cleaner and easier to understand/grasp. Not really a fan of the second CLI command option, doesn't lend itself well to being understood by editors since it's a custom format and requires specialized parsing logic if you want to process it instead of using something standardized.

Change 704845 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[cloud/toolforge/jobs-framework-cli@master] toolforge-jobs: add new option to load jobs from a YAML file

https://gerrit.wikimedia.org/r/704845

Change 704845 merged by Arturo Borrero Gonzalez:

[cloud/toolforge/jobs-framework-cli@master] toolforge-jobs: add new option to load jobs from a YAML file

https://gerrit.wikimedia.org/r/704845