Page MenuHomePhabricator

[maintain-kubeusers] Increment default services quota
Open, HighPublic

Description

For T348758: [jobs-api,jobs-cli] Support services in jobs we should increment the default services quota from the current 1 to some TBD higher value.

Event Timeline

taavi triaged this task as High priority.Apr 15 2024, 11:06 AM
taavi created this task.

to some TBD higher value.

How about 16 to match the number of pods in the default quota? We can certainly pick any other arbitrary number >1 as well, but this at least has some rationale.

I find it pretty unlikely that a typical tool will exhaust either the Pod or Service quota at 16; I think the most typical tool will continue to be a webservice that consumes one Pod and one Service. In the spirit of T306324: Consider improving quota workflow I think we should set the default limits quite high relative to expected use when possible so that we don't end up discouraging folks from innovating by making them stop to ask for permission to use the platform.

to some TBD higher value.

How about 16 to match the number of pods in the default quota? We can certainly pick any other arbitrary number >1 as well, but this at least has some rationale.

I find it pretty unlikely that a typical tool will exhaust either the Pod or Service quota at 16; I think the most typical tool will continue to be a webservice that consumes one Pod and one Service. In the spirit of T306324: Consider improving quota workflow I think we should set the default limits quite high relative to expected use when possible so that we don't end up discouraging folks from innovating by making them stop to ask for permission to use the platform.

I thought the plan is to only allow services for continuous jobs (which web-service will probably become entangled with soon)? if that's the case then we should make services quota the same as deployments quota which is currently 3. We can also increase this but atleast both services and deployments quotas should be in lockstep

I thought the plan is to only allow services for continuous jobs (which web-service will probably become entangled with soon)? if that's the case then we should make services quota the same as deployments quota which is currently 3. We can also increase this but atleast both services and deployments quotas should be in lockstep

$ kubectl describe quota
Name:                   tool-wikibugs
Namespace:              tool-wikibugs
Resource                Used    Hard
--------                ----    ----
configmaps              2       10
count/cronjobs.batch    0       50
count/deployments.apps  6       6
count/jobs.batch        0       15
limits.cpu              3       8
limits.memory           3Gi     8Gi
persistentvolumeclaims  0       0
pods                    6       16
requests.cpu            1375m   4
requests.memory         1536Mi  4Gi
secrets                 21      64
services                2       2
services.nodeports      0       0

I guess wikibugs has had a quota bump for this already (count/deployments.apps 6) and is currently bumping up against the raised limit too. Why do we care how many Deployments a namespace has as long as the CPU and RAM quotas allow the Pods they manage?

Very valid point Bryan. I can't think of a reason too. The most important quotas to put in place imo are hardware resource quotas like ram and cpu. Maybe @dcaro and @taavi has an opinion?

We need to have some quota in place to prevent a misbehaving tool from taking kube-apiserver down by creating hundreds or thousands of unfillable ReplicaSets (a similar thing has happened in the past, see T301081), but I have no objections to raising the deployment quota to match the pod one for example.