Quota usage not being counted properly in new region
Open, Needs TriagePublic

Description

Horizon and openstack-browser are showing a usage of 9 instances for deployment-prep in eqiad1-r. It should be more like ~68. I imagine the VCPU/RAM calculations are off too. I believe this data comes from the Nova service itself.

Krenair created this task.Fri, Nov 23, 12:55 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFri, Nov 23, 12:55 AM
Krenair updated the task description. (Show Details)Fri, Nov 23, 12:56 AM
Krenair added a subscriber: Andrew.
Krenair updated the task description. (Show Details)Fri, Nov 23, 1:01 AM

I believe floating IPs and security groups are a different problem - in eqiad1-r this should be from the neutron service instead. Will open a separate task about that.

Krenair updated the task description. (Show Details)Thu, Dec 6, 11:13 PM
bd808 added a subscriber: bd808.Tue, Dec 11, 11:59 PM

Found upstream at https://bugs.launchpad.net/nova/+bug/1742826/comments/4:

There is a well-known issue with quotas "going out of sync" in Nova versions Ocata and earlier and is why the 'nova-manage project quota_usage_refresh' command existed. Quotas out-of-sync means that the quota_usages do not match the actual resources being consumed. This can occur due to races while restarting nova-compute, etc.

I also found https://specs.openstack.org/openstack/nova-specs/specs/mitaka/approved/refresh-quotas-usage.html and https://blueprints.launchpad.net/nova/+spec/refresh-quotas-usage. I think this "fix" missed mitaka, landed in newton, and was made obsolete in pike, but I'm not 100% certain. I definitely can't find a working nova-manage project quota_usage_refresh or similar on cloudcontrol1003.