In T196486: Concurrent generated jobs from a single user overloaded grid engine we found that a well intentioned user had started hundreds of parallel jobs on the grid. We added a specific configuration to keep that particular user from doing this again. It would be nice however to have a better circuit breaker to stop this from happening again easily.
The maximum number of jobs any user may have running in a Sun Grid Engine cluster at the same time. If set to 0 (default) the users may run an arbitrary number of jobs.
We currently have this set in the grid config with a value of 1000 which coincidentally(?) is also the upper limit on jobs per queue. This means that the limit functionally will never take effect.