Currently our namespace ResourceQuota and LimitRanger configs are not very flexible.
For example we currently don't support running containers without limits (as all container in a namespace with ResourceQuota set do require defined requests and limits).
We should refactor the helmfile.d code to support more use cases like namespaces without quota/limitrange.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Stalled | None | T255792 Quibble runs core:unit tests twice! | |||
| Open | None | T328919 Upgrade to PHPUnit 10 | |||
| Open | None | T338103 Micro-optimize ApiResult::isMetadataKey with str_starts_with once we support PHP8+ | |||
| Open | None | T328921 Drop PHP 7.4 support from MediaWiki | |||
| Stalled | None | T334726 Use return type `never` in Wikibase | |||
| Open | None | T328922 Drop PHP 8.0 support from MediaWiki | |||
| Stalled | None | T319055 Upgrade to psr/container 2.x | |||
| Stalled | Krinkle | T319432 Migrate WMF production from PHP 7.4 to PHP 8.1 | |||
| Open | None | T291916 Tracking task for Bullseye migrations in production | |||
| Stalled | None | T356293 Migrate MW appservers' base images to bullseye | |||
| Open | None | T290536 Serve production traffic via Kubernetes | |||
| Resolved | Clement_Goubert | T342748 mw-on-k8s app container CPU throttling at low average load | |||
| Resolved | JMeybohm | T343978 Allow more flexibility in ResourceQuota and LimitRanger config |
Change 947865 had a related patch set uploaded (by JMeybohm; author: JMeybohm):
[operations/deployment-charts@master] CI: Bail out if admin_ng build fails completely
Change 947866 had a related patch set uploaded (by JMeybohm; author: JMeybohm):
[operations/deployment-charts@master] admin_ng: Add more configuration options for resourcequota and limits
Change 948128 had a related patch set uploaded (by JMeybohm; author: JMeybohm):
[operations/deployment-charts@master] Remove limits in ResourceQuota and container limitanges for mediawiki
Change 947865 merged by jenkins-bot:
[operations/deployment-charts@master] CI: Bail out if admin_ng build fails completely
Change 947866 merged by jenkins-bot:
[operations/deployment-charts@master] admin_ng: Add more configuration options for resourcequota and limits
Change 948128 merged by jenkins-bot:
[operations/deployment-charts@master] Remove limits in ResourceQuota and container limitanges for mediawiki
Deployed, we'll be testing it with new deployments for T342748, I will reopen if we encounter any issue.
Reverting https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/948128
3m7s Warning FailedCreate replicaset/mw-web.eqiad.canary-77cf597c89 Error creating: pods "mw-web.eqiad.canary-77cf597c89-7m7pg" is forbidden: [minimum cpu usage per Pod is 100m. No request is specified, minimum memory usage per Pod is 100Mi. No request is specified, maximum cpu usage per Pod is 9. No limit is specified, maximum memory usage per Pod is 5Gi. No limit is specified] 3m7s Warning FailedCreate replicaset/mw-web.eqiad.canary-77cf597c89 Error creating: pods "mw-web.eqiad.canary-77cf597c89-ndh7q" is forbidden: [minimum cpu usage per Pod is 100m. No request is specified, minimum memory usage per Pod is 100Mi. No request is specified, maximum cpu usage per Pod is 9. No limit is specified, maximum memory usage per Pod is 5Gi. No limit is specified] 3m7s Warning FailedCreate replicaset/mw-web.eqiad.canary-77cf597c89 Error creating: pods "mw-web.eqiad.canary-77cf597c89-7bgbj" is forbidden: [minimum cpu usage per Pod is 100m. No request is specified, minimum memory usage per Pod is 100Mi. No request is specified, maximum cpu usage per Pod is 9. No limit is specified, maximum memory usage per Pod is 5Gi. No limit is specified] 3m7s Warning FailedCreate replicaset/mw-web.eqiad.canary-77cf597c89 Error creating: pods "mw-web.eqiad.canary-77cf597c89-f78g4" is forbidden: [minimum cpu usage per Pod is 100m. No request is specified, minimum memory usage per Pod is 100Mi. No request is specified, maximum cpu usage per Pod is 9. No limit is specified, maximum memory usage per Pod is 5Gi. No limit is specified] 3m7s Warning FailedCreate replicaset/mw-web.eqiad.canary-77cf597c89 Error creating: pods "mw-web.eqiad.canary-77cf597c89-c7298" is forbidden: [minimum cpu usage per Pod is 100m. No request is specified, minimum memory usage per Pod is 100Mi. No request is specified, maximum cpu usage per Pod is 9. No limit is specified, maximum memory usage per Pod is 5Gi. No limit is specified] 3m7s Warning FailedCreate replicaset/mw-web.eqiad.canary-77cf597c89 Error creating: pods "mw-web.eqiad.canary-77cf597c89-459n5" is forbidden: [minimum cpu usage per Pod is 100m. No request is specified, minimum memory usage per Pod is 100Mi. No request is specified, maximum cpu usage per Pod is 9. No limit is specified, maximum memory usage per Pod is 5Gi. No limit is specified] 3m7s Warning FailedCreate replicaset/mw-web.eqiad.canary-77cf597c89 Error creating: pods "mw-web.eqiad.canary-77cf597c89-552h7" is forbidden: [minimum cpu usage per Pod is 100m. No request is specified, minimum memory usage per Pod is 100Mi. No request is specified, maximum cpu usage per Pod is 9. No limit is specified, maximum memory usage per Pod is 5Gi. No limit is specified] 3m6s Warning FailedCreate replicaset/mw-web.eqiad.canary-77cf597c89 Error creating: pods "mw-web.eqiad.canary-77cf597c89-swkqc" is forbidden: [minimum cpu usage per Pod is 100m. No request is specified, minimum memory usage per Pod is 100Mi. No request is specified, maximum cpu usage per Pod is 9. No limit is specified, maximum memory usage per Pod is 5Gi. No limit is specified] 3m6s Warning FailedCreate replicaset/mw-web.eqiad.canary-77cf597c89 Error creating: pods "mw-web.eqiad.canary-77cf597c89-dj2x2" is forbidden: [minimum memory usage per Pod is 100Mi. No request is specified, minimum cpu usage per Pod is 100m. No request is specified, maximum cpu usage per Pod is 9. No limit is specified, maximum memory usage per Pod is 5Gi. No limit is specified] 23s Warning FailedCreate replicaset/mw-web.eqiad.canary-77cf597c89 (combined from similar events): Error creating: pods "mw-web.eqiad.canary-77cf597c89-mwdt8" is forbidden: [minimum cpu usage per Pod is 100m. No request is specified, minimum memory usage per Pod is 100Mi. No request is specified, maximum cpu usage per Pod is 9. No limit is specified, maximum memory usage per Pod is 5Gi. No limit is specified] 3m7s Normal ScalingReplicaSet deployment/mw-web.eqiad.canary Scaled up replica set mw-web.eqiad.canary-77cf597c89 to 1 5m47s Warning Unhealthy pod/mw-web.eqiad.main-7bd65dbfd-t4djj Readiness probe failed: command "/bin/healthz" timed out
minimum cpu usage per Pod is 100m. No request is specified, minimum memory usage per Pod is 100Mi. No request is specified
This is because the exporter containers lack resource requests.
maximum cpu usage per Pod is 9. No limit is specified, maximum memory usage per Pod is 5Gi. No limit is specified
This is because we still have Pod limits set in LimitRanger.
Change 950068 had a related patch set uploaded (by JMeybohm; author: JMeybohm):
[operations/deployment-charts@master] Revert "Revert "Remove limits in ResourceQuota and container limitanges for mediawiki""
Change 950068 merged by jenkins-bot:
[operations/deployment-charts@master] Revert "Revert "Remove limits in ResourceQuota and container limitanges for mediawiki""
Change 951050 had a related patch set uploaded (by JMeybohm; author: JMeybohm):
[operations/deployment-charts@master] admin_ng: Lower the minimum pod memory request to 50Mi
Change 951050 merged by jenkins-bot:
[operations/deployment-charts@master] admin_ng: Lower the minimum pod memory request to 50Mi
Change 951055 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):
[operations/deployment-charts@master] admin_ng: Raise max cpu per pod to 10
Change 951055 merged by jenkins-bot:
[operations/deployment-charts@master] admin_ng: Raise max cpu per pod to 10
Change 951059 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):
[operations/deployment-charts@master] admin_ng: Lower the minimum container memory request to 50Mi
Change 951059 merged by jenkins-bot:
[operations/deployment-charts@master] admin_ng: Lower the minimum container memory request to 50Mi