Instead of having a home grown pile of python scripts, what can we use as a building block for a job scheduler and manager?
|Stalled||None||T146070 Review progress of dumps rewrite|
|Open||None||T128513 Dumps 2.0 Platform design questions|
|Open||None||T128520 What already available software can we build on for a job scheduler?|
|Open||None||T143205 Draft usage scenarios for job/workflow manager|
|Open||None||T143206 List requirements needed for task/job/workflow manager.|
|Open||None||T143207 Evaluate software packages for job/task/workflow management|
Time to get started on this. If done properly it can be used for production of the current dumps as well, allowing us to remove several levels of annoying abstraction from the existing code.
- Draft several usage scenarios for task submission, status check requests and more
- Set out requirements for task manager/job scheduler/workflow manager
- Evaluate existing software
- Choose one!
Guess I'll turn these into subtasks so we can start tossing notes into them.
This task has been assigned to the same task owner for more than two years. Resetting task assignee due to inactivity, to decrease task cookie-licking and to get a slightly more realistic overview of plans. Please feel free to assign this task to yourself again if you still realistically work or plan to work on this task - it would be welcome!
For tips how to manage individual work in Phabricator (noisy notifications, lists of task, etc.), see https://phabricator.wikimedia.org/T228575#6237124 for available options.
(For the records, two emails were sent to assignee addresses before resetting assignees. See T228575 for more info and for potential feedback. Thanks!)