Change Details

The ORES service is fairly experimental / researchy in nature, and uses significant resources on the SCB cluster. Currently, we don't enforce resource limits, which has led to several instances of ORES using all RAM, cpu or disk space on SCB hosts. At least in some of these instances this directly led to outages of other SCB services. To avoid a repeat, we should set up hard resource limits for ORES. The most pressing resources are RAM and disk quota. CPU priority is likely already taken care of by default systemd cgroup behavior. Looking at http://0pointer.de/blog/projects/resources.html, limiting memory usage should not be much more effort than adding something like ``` [Service] MemoryLimit=1G ``` Setting up disk quotas looks a bit harder. An alternative to native user quotas might be to move ORES storage & logging to a dedicated partition.

The ORES service is fairly experimental / researchy in nature, and uses significant resources on the SCB cluster. Currently, we don't enforce resource limits. We should set up hard resource limits for ORES. The most pressing resources are RAM and disk quota. CPU priority is likely already taken care of by default systemd cgroup behavior. Looking at http://0pointer.de/blog/projects/resources.html, limiting memory usage should not be much more effort than adding something like ``` [Service] MemoryLimit=1G ``` Setting up disk quotas looks a bit harder. An alternative to native user quotas might be to move ORES storage & logging to a dedicated partition.

The ORES service is fairly experimental / researchy in nature, and uses significant resources on the SCB cluster. Currently, we don't enforce resource limits, which has led to several instances of ORES using all RAM, cpu or disk space on SCB hosts. At least in some of these instances this directly led to outages of other SCB services. To avoid a repeat, weWe should set up hard resource limits for ORES. The most pressing resources are RAM and disk quota. CPU priority is likely already taken care of by default systemd cgroup behavior. Looking at http://0pointer.de/blog/projects/resources.html, limiting memory usage should not be much more effort than adding something like ``` [Service] MemoryLimit=1G ``` Setting up disk quotas looks a bit harder. An alternative to native user quotas might be to move ORES storage & logging to a dedicated partition.