With jsub, you could use -e and -o to redirect stdout and stderr to some custom files (e.g. the same file for both). This is not possible when using the newer toolforge-jobs command, which will always use jobname.out and jobname.err.
Description
Description
Details
Details
Customize query in gerrit
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | • JHedden | T251027 "signatures" tool has failed job pods on Kubernetes cluster | |||
| Resolved | aborrero | T251917 Design the Jobs service in k8s | |||
| Resolved | aborrero | T283238 Toolforge: develop jobs-framework-api | |||
| Resolved | aborrero | T285944 Toolforge: beta phase for the new jobs framework | |||
| Resolved | • Bstorm | T126083 overhaul labstore setup [tracking] | |||
| Resolved | • GTirloni | T216988 labstore1004 - DISK CRITICAL - free space: /srv/tools 115904 MB (1% inode=79%): | |||
| Resolved | • Bstorm | T217993 2019-03-10: tools and NFS share cleanup (high usage) | |||
| Resolved | • Bstorm | T122508 Prevent overly-large log files | |||
| Declined | None | T286847 Add webservice flag to mount project directory read-only | |||
| Open | None | T127367 [toolforge,jobs-api,webservice,storage] Provide modern, non-NFS log solution for Toolforge webservices and bots | |||
| Duplicate | Feature | Raymond_Ndibe | T304421 Allow customizing the out/err files with toolforge-jobs |
Event Timeline
Comment Actions
It does feel like this creates some kind of conflict with T301901 which already have two patches addressing it.
- should this ( --out and --err) take precedence over --logdir being introduced by T301901 ?
- What happens when one of --out or --err is not explicitly provided? my assumption is that we fallback to whatever path supplied by --logdir if present, or to the default 1>>{jobname}.out 2>>{jobname}.err
Comment Actions
Is --logdir documented somewhere? The online man pages does not show it. Option -help does not show it too.
Its hard to decide any precedence when there is nothing to decide on.
Comment Actions
Not yet documented but we currently have two patches pending review that introduces --logdir. And since there is already some kind of consensus regarding the introduction of --logdir (no one is yet to prove that it is not a necessary change), it is reasonable to conclude that it is only are matter of time before this is satisfactorily reviewed and merged. In that sense, it makes sense to talk about precedence of the coming --logdir relative to T304421 and T302211
Comment Actions
Okay.
For me(!) --logdir would be enough. I just want to have a (relative) clean home directory.
To clarify: I have currently ~50 scripts, a few are running day by day, the others run once per week or once per month. So without any option for redirecting I would end up with 100 additional files in the home-directory.
Comment Actions
Yes, what bothers me is maybe providing too many options for customizing the handling of log files it too much overload?
perhaps --log-dir is enough? if so then maybe we should close this task. T302211 appears distinct enough so should be left as it is. what are your thoughts on this @bd808?
I feel we should probably close this if we are not trying to address a specific usecase with it and just focus on T302211 and T301901 . this is because the difference between T301901 and this task is just a matter of deciding on what level of granular control users should be given over controlling the destination of stdout and stderr logs. Making up up our minds on that decision invalidates either T301901 or this task.
@Daimona was this task created because you need to have this specific granular control over where stdout and stderr logs are stored, or can something like --log-dir be enough for your usecase?
Comment Actions
I do need the granular control over where to store stdout and stderr. It's not a mission-critical feature, but it was there with jsub and it would come in handy sometimes.
Comment Actions
Change 827592 had a related patch set uploaded (by Raymond Ndibe; author: Raymond Ndibe):
[cloud/toolforge/jobs-framework-cli@master] jobs-framework-cli: add --stdout and --stderr args to cli
Comment Actions
Change 827594 had a related patch set uploaded (by Raymond Ndibe; author: Raymond Ndibe):
[cloud/toolforge/jobs-framework-api@main] jobs-framework-api: add --stdout and --stderr args to cli
Comment Actions
My proposal is that we leave the current filelog option as is. I think the investment that will really benefit us is trying to work on the root problem: T127367: [toolforge,jobs-api,webservice,storage] Provide modern, non-NFS log solution for Toolforge webservices and bots
Comment Actions
Change 867591 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):
[cloud/toolforge/jobs-framework-api@main] tjf: separate command management into its own file
Comment Actions
Change 868076 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):
[cloud/toolforge/jobs-framework-api@main] command: move log redirections to the wrapper sh call
Comment Actions
Change 867591 merged by jenkins-bot:
[cloud/toolforge/jobs-framework-api@main] tjf: separate command management into its own file
Comment Actions
Change 868076 merged by jenkins-bot:
[cloud/toolforge/jobs-framework-api@main] command: move log redirections to the wrapper sh call
Comment Actions
Change 871171 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):
[cloud/toolforge/jobs-framework-api@main] command: introduce filelog_std{out,err} parameters
Comment Actions
Change 827594 merged by jenkins-bot:
[cloud/toolforge/jobs-framework-api@main] jobs-framework-api: add filelog_stdout and filelog_stderr arguments
Comment Actions
Change 827592 merged by jenkins-bot:
[cloud/toolforge/jobs-framework-cli@master] jobs-framework-cli: add --filelog-stdout and --filelog-stderr args to cli