I am regularly receiving error messages in e-mail since two days from Cron Daemon about my scheduled jobs with the title like as follows:
Cron Daemon - Cron <tools.ato@tools-cron-01> jsub -once -N sign-of-life -mem 512m -o public_html/log/sign-of-life.txt -j y scripts/sign-of-life-core.sh > /dev/null
The content is the following:
/bin/sh: 1: cannot create /dev/null
: Permission denied
For the record, in SAL, 2016-01-25 20:30 @yuvipanda: switched over cron host to tools-cron-01, manually copied all old cron files from tools-submit to tools-cron-01
Odd:
valhallasw@tools-cron-01:~$ ls -l /dev/null crw-rw-rw- 1 root root 1, 3 Jan 21 19:44 /dev/null valhallasw@tools-cron-01:~$ echo "boo" > /dev/null
gerrit-reviewer-bot's cron also pipes to /dev/null, but works without issues.
It also works on the shell as tools.ato as well.
The tools.ato crontab does have \r\n line endings -- could that cause this? I'm also confused why this would suddenly give different behavior, but that might be a precise vs trusty issue.
edit: the \r being picked up as filename is consistent with
/bin/sh: 1: cannot create /dev/null : Permission denied
The dump at /data/project/.system/crontabs has no \rs, so the line endings must have been introduced between:
scfc@tools-cron-01:~$ sudo fgrep -l '^M' /data/project/.system/crontabs/tools-cron-01.tools.eqiad.wmflabs/tools.ato /var/spool/cron/crontabs/tools.ato /var/spool/cron/crontabs/tools.ato scfc@tools-cron-01:~$ sudo ls -l /data/project/.system/crontabs/tools-cron-01.tools.eqiad.wmflabs/tools.ato /var/spool/cron/crontabs/tools.ato -r--r----- 1 root tools.admin 2689 Jan 25 20:45 /data/project/.system/crontabs/tools-cron-01.tools.eqiad.wmflabs/tools.ato -rw------- 1 tools.ato crontab 2966 Jan 26 22:12 /var/spool/cron/crontabs/tools.ato scfc@tools-cron-01:~$
This would be consistent with changes to the content of the crontab (cf. sudo diff -uw /data/project/.system/crontabs/tools-cron-01.tools.eqiad.wmflabs/tools.ato /var/spool/cron/crontabs/tools.ato).
So there would be no changes in the system, but in the crontab.
However I always assumed that cron feeds the line to a shell which then splits it into words, etc., and I expected that to treat \r the same way as any other white space. I'll take another look into the source.
Thank you all!
The problem, I did is solved.
I think I used the text editor of WinSCP and it was not the best choice. :D
cron does indeed just feed the line to the shell, but there's the problem:
scfc@tools-cron-01:~$ bash -c 'echo Test > /dev/null^M' : Permission denied scfc@tools-cron-01:~$
bash(1) says under "Shell Variables":
IFS: The Internal Field Separator that is used for word splitting after expansion and to split lines into words with the read builtin command. The default value is ``<space><tab><newline>''.
So carriage return (\r) is not a word separator by default and thus the line gets split into /dev/null\r.