This is urgent, blocking work so please help limit scope. This task is finished when the following features are complete:
process-control
-----
* [x] Ops can package and deploy the tool.
* [x] stdout logfiles are written one file per job run. (T161155)
* [x] Devs can run jobs one-off.
* [x] Runs jobs according to a code-generated crontab.
* [x] Never drop logs even (especially!) if the process is killed unexpectedly. (T161571)
* [ ] Failmail when job exits with non-zero return code--let not perfect be thine enemy.
* [ ] Working workaround for specific chained jobs. (T161035)
puppet
-----
* [ ] Jobs configuration is sync'ed to /var/lib/process-control with localsettings. Read-only.
* [x] Global configuration file is synced to /etc/process-control.yaml
* [x] Devs have sudo access to the scripts and can pass any CLI params.
* [x] jenkins g+ws /var/log/process-control
* [ ] cron-generate can somehow write to /etc/cron.d/process-control
* [ ] Nobody can accidentally run the script as their own user.
Not in MVP scope
-----
* Devs can kill jobs.
* Log actions and errors to syslog.
* script to list all jobs and statuses (T161584)
* should be able to disable groups of jobs (T160699)
* repeated failure handling (T161567)
* Turn process-control lock module into a context manager (T161536)
* Clean up deb packaging once we're on Jessie.