Page MenuHomePhabricator
Create Task
Maniphest T330759

Modernize openstack rbac
Closed, ResolvedPublic
Actions

Description

codfw1-dev:

  • replace 'observer' roles with 'reader' roles
  • replace 'user' roles with 'reader' roles
  • replace 'projectadmin' role with 'member' role
  • remove old 'user' and 'projectadmin' roles

eqiad1:

  • replace 'observer' roles with 'reader' roles
  • replace 'user' roles with 'reader' roles
  • replace 'projectadmin' role with 'member' role (?)
  • rename old 'user' and 'projectadmin' roles (preserves a revert option)
  • remove old 'user' and 'projectadmin' roles

all together:

  • enforce_scope = true
  • enforce_new_defaults = true
  • remove projectadmin-specific policy rules
  • remove observer-specific policy rules
  • review and remove as many custom policy rules as possible
  • replace most or all uses of 'novaadmin' with service roles with global admin

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Openstack Nova: use 'novaservice' service user rather than novaadminoperations/puppetproduction+8 -8
Openstack Nova: use 'novaservice' service user rather than novaadminoperations/puppetproduction+5 -5
Added stand-in passwords for nova service userlabs/privatemaster+2 -0
cinder: use 'cinder' service user rather than 'novaadmin'operations/puppetproduction+6 -6
Openstack [service_user] config: use internal endpoint for service usersoperations/puppetproduction+5 -2
Openstack [keystone_authtoken]: remove auth_url settingoperations/puppetproduction+0 -1
cinder: use 'cinder' service user rather than 'novaadmin'operations/puppetproduction+2 -2
Remove oslo_policy sectionoperations/puppetproduction+0 -7
Designate: update policy.yamloperations/puppetproduction+15 -39
Neutron: update policy rulesoperations/puppetproduction+3 -33
cinder policy.yaml: update, remove redundant rulesoperations/puppetproduction+0 -25
Cinder: explicitly use new policy rulesoperations/puppetproduction+4 -0
glance: update policy.yamloperations/puppetproduction+0 -8
nova policy.json: remove a bunch of redundant rulesoperations/puppetproduction+0 -31
nova policy.yaml: update with advice from oslopolicy-validatoroperations/puppetproduction+40 -51
keystone: update policy.yaml filesoperations/puppetproduction+8 -14
cloud-vps: clean up a couple of hiera settingsoperations/puppetproduction+369 -579
eqiad1: enforce_new_policy_defaults: Trueoperations/puppetproduction+1 -1
codfw1dev backups: enforce_policy_scope: trueoperations/puppetproduction+1 -1
codfw1dev: enforce_new_policy_defaults=trueoperations/puppetproduction+1 -1
eqiad1 openstack: enforce_policy_scope=Trueoperations/puppetproduction+1 -1
Openstack codfw1dev: enforce_policy_scope: trueoperations/puppetproduction+1 -1
OpenStack: add a clouds.yaml file for environment setupoperations/puppetproduction+68 -8
grid_configurator: use mwopenstackclients libraryoperations/puppetproduction+15 -65
wmcs prometheus: include 'OPENSTACK->CLOUD' in prometheus configoperations/puppetproduction+2 -0
clouds.yaml: remove keystoneadmin sectionoperations/puppetproduction+0 -19
enforce_new_policy_defaults: false in codfw1devoperations/puppetproduction+1 -1
enforce_policy_scope: false in codfw1devoperations/puppetproduction+1 -1
clouds.yaml: and yet still further fixes to support system scope sectionsoperations/puppetproduction+1 -1
clouds.yaml: yet further fixes to support system scope sectionsoperations/puppetproduction+3 -0
clouds.yaml: further fixes to support system scope sectionsoperations/puppetproduction+2 -1
clouds.yaml: specify system_scope for special system sectionsoperations/puppetproduction+19 -2
neutron policy.yaml: don't override network_device policyoperations/puppetproduction+0 -1
codfw1dev: enforce scope and default policiesoperations/puppetproduction+2 -2
mwopenstackclient: better support projectless authoperations/puppetproduction+26 -22
codfw1dev: enforce scope and default policiesoperations/puppetproduction+2 -2
toolforge k8s: permit access to clouds.yamloperations/puppetproduction+4 -0
nfs-exportd service: subscribe to clouds.yamloperations/puppetproduction+1 -1
graphite archive-instances.py: use mwopenstackclient for openstack accessoperations/puppetproduction+6 -45
nfs-exportd: convert to using mwopenstackclients and --os-cloudoperations/puppetproduction+25 -73
wmcs-spreadcheck: use clouds.yaml section rather than envfileoperations/puppetproduction+3 -3
wmcs notify_maintainers: use mwopenstackclients for keystoneclientoperations/puppetproduction+2 -21
wmcs-enc-cli: use os-cloud section rather than envfileoperations/puppetproduction+4 -4
wmcs-wikireplica-dns: use os-cloud section rather than envfileoperations/puppetproduction+5 -5
wmcs-webproxy: use os-cloud section rather than envfileoperations/puppetproduction+8 -8
wmcs-dns-floating-ip-updater: use os-cloud section rather than envfileoperations/puppetproduction+6 -6
observerenv: include in both global and root-local clouds.yamloperations/puppetproduction+28 -23
envscript: mirror any global cloud definitions into root's clouds.yamloperations/puppetproduction+10 -0
OpenStack observerenv: Add global clouds.yaml file with observer credsoperations/puppetproduction+12 -0
openstack: envscript: do not set a default for clouds_fileoperations/puppetproduction+24 -25
OpenStack env scripts: different script names for new env scriptsoperations/puppetproduction+2 -2
Openstack envscripts.pp: create additional scripts for system and domain scopeoperations/puppetproduction+36 -0
openstack::util::envscript: allow caller to specify domain_idsoperations/puppetproduction+4 -4
Openstack envscripts: allow unsetting environment variablesoperations/puppetproduction+5 -1
codfw1dev: set enforce_policy_scope and enforce_new_policy_defaults to false.operations/puppetproduction+4 -2
Added dummy ldap_os_system_passlabs/privatemaster+2 -0
OpenStack Designate: role back codfw1dev change to default policiesoperations/puppetproduction+7 -2
Pass enforce_policy_scope and enforce_new_policy_defaults to cinder-backupoperations/puppetproduction+77 -61
glance: set enforce_secure_rbacoperations/puppetproduction+2 -0
OpenStack: adopt new scoped tokens and policy rules in codfw1devoperations/puppetproduction+512 -360
vps.add_user_to_project: update role names.cloud/wmcs-cookbooksmain+8 -8
striker: Bump container version to 2023-03-10-212005-productionoperations/puppetproduction+2 -2
Revert "striker: Bump container version to 2023-03-09-005633-production"operations/puppetproduction+2 -2
striker: Bump container version to 2023-03-09-005633-productionoperations/puppetproduction+2 -2
openstack: Update Keystone role nameslabs/strikermaster+32 -20
Remove use of the overloaded term 'member' in the sudo UIopenstack/horizon/wmf-sudo-dashboardmain+2 -2
Rename the 'Project Members' panel to 'Project Access'openstack/horizon/wmf-member-dashboardmain+3 -3
Rename 'projectadmin' role to 'member'openstack/horizon/wmf-member-dashboardmain+16 -15
OpenStack: rename 'projectadmin' role to 'member' roleoperations/puppetproduction+136 -137
OpenStack: collapse 'user' OpenStack role into 'reader' roleoperations/puppetproduction+45 -42
OpenStack: rename 'user' role to 'member'operations/puppetproduction+10 -10
cinder policy.yaml: redefine xena_system_admin_or_project_member ruleoperations/puppetproduction+5 -0
Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
gerritbot added a comment.May 7 2023, 1:33 PM

Change 916876 merged by Andrew Bogott:

[operations/puppet@production] mwopenstackclient: better support projectless auth

https://gerrit.wikimedia.org/r/916876

gerritbot added a comment.May 8 2023, 5:56 PM

Change 917390 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] codfw1dev: enforce scope and default policies

https://gerrit.wikimedia.org/r/917390

gerritbot added a comment.May 8 2023, 6:09 PM

Change 917390 merged by Andrew Bogott:

[operations/puppet@production] codfw1dev: enforce scope and default policies

https://gerrit.wikimedia.org/r/917390

gerritbot added a comment.May 8 2023, 6:52 PM

Change 917396 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] neutron policy.yaml: don't override network_device policy

https://gerrit.wikimedia.org/r/917396

gerritbot added a comment.May 8 2023, 6:53 PM

Change 917396 merged by Andrew Bogott:

[operations/puppet@production] neutron policy.yaml: don't override network_device policy

https://gerrit.wikimedia.org/r/917396

gerritbot added a comment.May 10 2023, 2:12 AM

Change 917986 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] clouds.yaml: specify system_scope for special system sections

https://gerrit.wikimedia.org/r/917986

gerritbot added a comment.May 10 2023, 2:33 AM

Change 917986 merged by Andrew Bogott:

[operations/puppet@production] clouds.yaml: specify system_scope for special system sections

https://gerrit.wikimedia.org/r/917986

gerritbot added a comment.May 10 2023, 2:37 AM

Change 917993 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] clouds.yaml: further fixes to support system scope sections

https://gerrit.wikimedia.org/r/917993

gerritbot added a comment.May 10 2023, 2:41 AM

Change 917993 merged by Andrew Bogott:

[operations/puppet@production] clouds.yaml: further fixes to support system scope sections

https://gerrit.wikimedia.org/r/917993

gerritbot added a comment.May 10 2023, 2:46 AM

Change 917994 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] clouds.yaml: yet further fixes to support system scope sections

https://gerrit.wikimedia.org/r/917994

gerritbot added a comment.May 10 2023, 2:48 AM

Change 917994 merged by Andrew Bogott:

[operations/puppet@production] clouds.yaml: yet further fixes to support system scope sections

https://gerrit.wikimedia.org/r/917994

gerritbot added a comment.May 10 2023, 2:49 AM

Change 917995 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] clouds.yaml: and yet still further fixes to support system scope sections

https://gerrit.wikimedia.org/r/917995

gerritbot added a comment.May 10 2023, 2:50 AM

Change 917995 merged by Andrew Bogott:

[operations/puppet@production] clouds.yaml: and yet still further fixes to support system scope sections

https://gerrit.wikimedia.org/r/917995

Andrew mentioned this in T336379: Openstack API slowdowns.May 10 2023, 1:40 PM
Andrew added a comment.May 22 2023, 7:40 PM

https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rbac.html#the-issues-we-are-facing-with-scope-concept <- implies that enabling scope now is premature and may never be necessary. I'm torn between mourning the lost effort and cheering the simpler model

Andrew updated the task description. (Show Details)May 22 2023, 7:40 PM
gerritbot added a comment.May 24 2023, 4:24 PM

Change 922899 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] enforce_policy_scope: false in codfw1dev

https://gerrit.wikimedia.org/r/922899

gerritbot added a comment.May 24 2023, 4:27 PM

Change 922899 merged by Andrew Bogott:

[operations/puppet@production] enforce_policy_scope: false in codfw1dev

https://gerrit.wikimedia.org/r/922899

gerritbot added a comment.May 24 2023, 4:46 PM

Change 922904 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] enforce_new_policy_defaults: false in codfw1dev

https://gerrit.wikimedia.org/r/922904

gerritbot added a comment.May 24 2023, 4:47 PM

Change 922904 merged by Andrew Bogott:

[operations/puppet@production] enforce_new_policy_defaults: false in codfw1dev

https://gerrit.wikimedia.org/r/922904

Andrew added a comment.May 26 2023, 7:07 PM

The openstack rbac work that I've been doing[0] has hit some serious roadbumps, but I'm swiftly approaching a stopping point. Y'all are long overdue for an update, so here's a summary of where I'm at.

I've been trying to catch up with the official upstream recommendations about roles and scope[1]. Those docs suggest a new two-dimensional auth model, where any given token has a 'role' (reader/member/admin) and a 'scope' (project/domain/system). Roles are progressive, (such that e.g. a member role has all the rights of a reader), but scope is not, so a system token can only do system things and NOT domain or project things.

That latter fact (that a system token can't do project-scoped work) has been making me (and anyone using codfw1dev) miserable as we can no longer have an auth model that allows one account and one token to do all the things we're used to doing.

I've recently been made aware of a different work-in-progress document on this topic [2] that reiterates that last point. EVERYONE turns out to hate the scope model because it breaks all existing 'one account to rule them all' workflows.

So! There are three basic topic areas here, which I'll take one at a time:

  • Role renames
  • clouds.yaml
  • Supporting token scope
  • Supporting the 'new policy defaults'

role renames

Upstream recommends that we have three basic user roles: reader, member, and admin. Mercifully, these roles mapped fairly closely to our old observer/user/admin role. I've renamed everything to conform with upstream, so now our cloud uses reader/member/admin just like upstream. That has gone fairly well, and Horizon already reflects the new names. Likely there are obsolete docs on wikitech using the old names but I've updated the things I can find.

TODO:

  • keep an eye out for doc references to the old role names (user/projectadmin) and search/replace (with reader/member)

clouds.yaml

We rely heavily on environment settings for OpenStack config and auth, mostly via the novaenv.sh script which sets up the environment.

The new/right way to set up config and auth is clouds.yaml. I've set up clouds.yaml in most places to duplicate the use of the existing env scripts. So, for instance, on a VM we have /etc/openstack/clouds.yaml with an 'observer' section that provides the same creds as observerenv.sh. On cloudcontrols we have ~root/.config/openstack/clouds.yaml with a 'novaobserver' section.
Going forward

  • it's best to not 'source novaenv.sh' and instead either add "--os-cloud novaadmin" to your commandlines, or 'export OS_CLOUD=novaobserver'.
  • setting things in your environment (e.g. OS_PROJECT_ID=xxx) won't work, as clouds.yaml takes precedence. Train your fingers (and rewrite code) to use openstack flags instead ("--project xxx").

I've already updated many of our workflows (mwopenstackclients, cookbooks, etc) to use clouds.yaml.

TODO:

  • train our fingers to use OS_CLOUD rather than novaenv.sh
  • Document new cli workflows T336670

token scope

It turns out that essentially all cloud admins worldwide hate the new scope model. Upstream devs are responding to this by leaving the scope implementation in place (for edge cases that require it, most significantly Openstack Ironic which we don't use) and otherwise declaring every API to work with a project token.

That means that in the long run this change will be largely moot for us. I'm going to roll back whatever work I've done to try to adopt the new model and we'll all just have to suffer through the deprecation warnings (which are now themselves deprecated) until release B or C when things stabilize.

TODO:

  • leave oslo_policy->enforce_scope = false for now (and possibly forever)
  • ignore policy deprecation warnings about scope

new policy defaults

The current policy defaults seem to assume enforce_scope = true; when I turn oslo_policy->enforce_new_defaults = True without enforce_scope turned I see a fair number of incoherent failures.

TODO:

  • leave oslo_policy->enforce_new_defaults = false for now, revisit after the scope rework finishes in B or C.
  • ignore policy deprecation warnings about new defaults, for now

Thank you all for your patience with the ever-shifting ground on this topic.

[0] https://phabricator.wikimedia.org/T330759

[1] https://docs.openstack.org/nova/latest/configuration/policy-concepts.html

[2] https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rbac.html#the-issues-we-are-facing-with-scope-concept

gerritbot added a comment.May 26 2023, 8:25 PM

Change 923696 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] clouds.yaml: remove keystoneadmin section

https://gerrit.wikimedia.org/r/923696

Andrew closed subtask T336670: openstack cli: clarify and document usage as Resolved.Jun 5 2023, 7:39 PM
gerritbot added a comment.Jun 12 2023, 4:11 PM

Change 923696 merged by Andrew Bogott:

[operations/puppet@production] clouds.yaml: remove keystoneadmin section

https://gerrit.wikimedia.org/r/923696

gerritbot added a comment.Jun 12 2023, 4:42 PM

Change 916590 merged by Andrew Bogott:

[operations/puppet@production] wmcs prometheus: include 'OPENSTACK->CLOUD' in prometheus config

https://gerrit.wikimedia.org/r/916590

gerritbot added a comment.Jun 12 2023, 4:43 PM

Change 916588 merged by Andrew Bogott:

[operations/puppet@production] grid_configurator: use mwopenstackclients library

https://gerrit.wikimedia.org/r/916588

Maintenance_bot removed a project: Patch-For-Review.Jun 12 2023, 5:10 PM
Andrew changed the task status from Open to Stalled.Jul 27 2023, 4:11 PM

This is stalled pending the upstream making some decisions.

fnegri moved this task from FY2022/2023-Q4 to Blocked on the cloud-services-team board.Jul 27 2023, 4:11 PM
fnegri edited projects, added cloud-services-team; removed cloud-services-team (FY2022/2023-Q4).
fnegri removed a project: Goal.
fnegri subscribed.Aug 28 2023, 5:50 PM
fnegri mentioned this in T341285: Upgrade cloud-vps openstack to version 'Antelope'.Aug 29 2023, 2:41 PM
Andrew closed subtask T337577: Replace use of openstack environment settings with clouds.yaml as Resolved.Jul 11 2024, 1:51 PM
Andrew reopened subtask T337577: Replace use of openstack environment settings with clouds.yaml as Open.Aug 19 2024, 3:40 PM
Andrew closed subtask T337577: Replace use of openstack environment settings with clouds.yaml as Resolved.
gerritbot added a comment.Feb 14 2025, 1:55 PM

Change #1119715 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Openstack codfw1dev: enforce_policy_scope: true

https://gerrit.wikimedia.org/r/1119715

gerritbot added a project: Patch-For-Review.Feb 14 2025, 1:55 PM
gerritbot added a comment.Feb 14 2025, 2:12 PM

Change #1119715 merged by Andrew Bogott:

[operations/puppet@production] Openstack codfw1dev: enforce_policy_scope: true

https://gerrit.wikimedia.org/r/1119715

Andrew added a comment.Feb 14 2025, 2:24 PM

The above patch didn't break any of my known tests. I guess now we wait a bit and see if anyone runs into codfw1dev access surprises.

Maintenance_bot removed a project: Patch-For-Review.Feb 14 2025, 2:31 PM
gerritbot added a comment.Apr 10 2025, 8:02 PM

Change #1135819 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] eqiad1 openstack: enforce_policy_scope=True

https://gerrit.wikimedia.org/r/1135819

gerritbot added a project: Patch-For-Review.Apr 10 2025, 8:02 PM

Change #1135820 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] codfw1dev: enforce_new_policy_defaults=true

https://gerrit.wikimedia.org/r/1135820

gerritbot added a comment.Apr 10 2025, 8:22 PM

Change #1135819 merged by Andrew Bogott:

[operations/puppet@production] eqiad1 openstack: enforce_policy_scope=True

https://gerrit.wikimedia.org/r/1135819

gerritbot added a comment.Apr 10 2025, 8:23 PM

Change #1135820 merged by Andrew Bogott:

[operations/puppet@production] codfw1dev: enforce_new_policy_defaults=true

https://gerrit.wikimedia.org/r/1135820

Andrew updated the task description. (Show Details)Apr 10 2025, 8:34 PM
gerritbot added a comment.May 2 2025, 5:31 PM

Change #1140767 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] codfw1dev backups: enforce_policy_scope: true

https://gerrit.wikimedia.org/r/1140767

gerritbot added a comment.May 2 2025, 5:31 PM

Change #1140768 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] eqiad1: enforce_new_policy_defaults: True

https://gerrit.wikimedia.org/r/1140768

gerritbot added a comment.May 2 2025, 5:42 PM

Change #1140767 merged by Andrew Bogott:

[operations/puppet@production] codfw1dev backups: enforce_policy_scope: true

https://gerrit.wikimedia.org/r/1140767

gerritbot added a comment.May 2 2025, 5:51 PM

Change #1140773 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] cloud-vps: clean up a couple of hiera settings

https://gerrit.wikimedia.org/r/1140773

gerritbot added a comment.May 5 2025, 12:20 PM

Change #1140768 merged by Andrew Bogott:

[operations/puppet@production] eqiad1: enforce_new_policy_defaults: True

https://gerrit.wikimedia.org/r/1140768

gerritbot added a comment.May 5 2025, 3:15 PM

Change #1140773 merged by Andrew Bogott:

[operations/puppet@production] cloud-vps: clean up a couple of hiera settings

https://gerrit.wikimedia.org/r/1140773

gerritbot added a comment.May 5 2025, 9:39 PM

Change #1141977 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] keystone: update policy.yaml files

https://gerrit.wikimedia.org/r/1141977

gerritbot added a comment.May 5 2025, 9:39 PM

Change #1141978 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] nova policy.yaml: update with advice from oslopolicy-validator

https://gerrit.wikimedia.org/r/1141978

gerritbot added a comment.May 5 2025, 9:39 PM

Change #1141979 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] nova policy.json: remove a bunch of redundant rules

https://gerrit.wikimedia.org/r/1141979

gerritbot added a comment.May 5 2025, 9:39 PM

Change #1141980 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] glance: update policy.yaml

https://gerrit.wikimedia.org/r/1141980

gerritbot added a comment.May 5 2025, 9:39 PM

Change #1141981 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Cinder: explicitly use new policy rules

https://gerrit.wikimedia.org/r/1141981

gerritbot added a comment.May 5 2025, 9:39 PM

Change #1141982 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] cinder policy.yaml: update, remove redundant rules

https://gerrit.wikimedia.org/r/1141982

gerritbot added a comment.May 5 2025, 9:39 PM

Change #1141983 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Neutron: update policy rules

https://gerrit.wikimedia.org/r/1141983

gerritbot added a comment.May 5 2025, 9:39 PM

Change #1141984 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Designate: update policy.yaml

https://gerrit.wikimedia.org/r/1141984

Andrew updated the task description. (Show Details)May 5 2025, 10:09 PM
taavi changed the task status from Stalled to Open.May 6 2025, 9:18 AM
gerritbot added a comment.May 6 2025, 2:07 PM

Change #1141977 merged by Andrew Bogott:

[operations/puppet@production] keystone: update policy.yaml files

https://gerrit.wikimedia.org/r/1141977

gerritbot added a comment.May 6 2025, 3:04 PM

Change #1141978 merged by Andrew Bogott:

[operations/puppet@production] nova policy.yaml: update with advice from oslopolicy-validator

https://gerrit.wikimedia.org/r/1141978

gerritbot added a comment.May 6 2025, 3:04 PM

Change #1141979 merged by Andrew Bogott:

[operations/puppet@production] nova policy.json: remove a bunch of redundant rules

https://gerrit.wikimedia.org/r/1141979

gerritbot added a comment.May 6 2025, 3:14 PM

Change #1141980 merged by Andrew Bogott:

[operations/puppet@production] glance: update policy.yaml

https://gerrit.wikimedia.org/r/1141980

gerritbot added a comment.May 6 2025, 3:14 PM

Change #1141981 merged by Andrew Bogott:

[operations/puppet@production] Cinder: explicitly use new policy rules

https://gerrit.wikimedia.org/r/1141981

gerritbot added a comment.May 6 2025, 3:14 PM

Change #1141982 merged by Andrew Bogott:

[operations/puppet@production] cinder policy.yaml: update, remove redundant rules

https://gerrit.wikimedia.org/r/1141982

gerritbot added a comment.May 6 2025, 3:14 PM

Change #1141983 merged by Andrew Bogott:

[operations/puppet@production] Neutron: update policy rules

https://gerrit.wikimedia.org/r/1141983

gerritbot added a comment.May 6 2025, 3:14 PM

Change #1141984 merged by Andrew Bogott:

[operations/puppet@production] Designate: update policy.yaml

https://gerrit.wikimedia.org/r/1141984

Maintenance_bot removed a project: Patch-For-Review.May 6 2025, 3:30 PM
gerritbot added a comment.May 6 2025, 8:27 PM

Change #1142683 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Remove oslo_policy section

https://gerrit.wikimedia.org/r/1142683

gerritbot added a project: Patch-For-Review.May 6 2025, 8:27 PM
gerritbot added a comment.May 6 2025, 8:33 PM

Change #1142683 merged by Andrew Bogott:

[operations/puppet@production] Remove oslo_policy section

https://gerrit.wikimedia.org/r/1142683

Andrew updated the task description. (Show Details)May 6 2025, 8:34 PM
gerritbot added a comment.May 8 2025, 4:11 PM

Change #1143612 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] cinder: use 'cinder' service user rather than 'novaadmin'

https://gerrit.wikimedia.org/r/1143612

gerritbot added a comment.May 9 2025, 1:57 PM

Change #1143612 abandoned by Andrew Bogott:

[operations/puppet@production] cinder: use 'cinder' service user rather than 'novaadmin'

Reason:

I don't think we're ready for this yet

https://gerrit.wikimedia.org/r/1143612

gerritbot added a comment.Jun 19 2025, 3:14 AM

Change #1161113 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Openstack [service_user] config: use internal endpoint for service users

https://gerrit.wikimedia.org/r/1161113

gerritbot added a comment.Jun 19 2025, 3:15 AM

Change #1161114 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Openstack [keystone_authtoken]: remove auth_url setting

https://gerrit.wikimedia.org/r/1161114

gerritbot added a comment.Jun 19 2025, 3:15 AM

Change #1161115 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] cinder: use 'cinder' service user rather than 'novaadmin'

https://gerrit.wikimedia.org/r/1161115

gerritbot added a comment.Jun 19 2025, 4:20 AM

Change #1161113 merged by Andrew Bogott:

[operations/puppet@production] Openstack [service_user] config: use internal endpoint for service users

https://gerrit.wikimedia.org/r/1161113

gerritbot added a comment.Jun 19 2025, 4:20 AM

Change #1161114 merged by Andrew Bogott:

[operations/puppet@production] Openstack [keystone_authtoken]: remove auth_url setting

https://gerrit.wikimedia.org/r/1161114

gerritbot added a comment.Jun 20 2025, 4:25 PM

Change #1161115 merged by Andrew Bogott:

[operations/puppet@production] cinder: use 'cinder' service user rather than 'novaadmin'

https://gerrit.wikimedia.org/r/1161115

gerritbot added a comment.Jun 20 2025, 6:51 PM

Change #1162060 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[labs/private@master] Added stand-in passwords for nova service user

https://gerrit.wikimedia.org/r/1162060

gerritbot added a comment.Jun 20 2025, 7:01 PM

Change #1162063 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Openstack Nova: use 'novaservice' service user rather than novaadmin

https://gerrit.wikimedia.org/r/1162063

gerritbot added a comment.Jun 20 2025, 7:08 PM

Change #1162060 merged by Andrew Bogott:

[labs/private@master] Added stand-in passwords for nova service user

https://gerrit.wikimedia.org/r/1162060

gerritbot added a comment.Jun 20 2025, 7:12 PM

Change #1162063 merged by Andrew Bogott:

[operations/puppet@production] Openstack Nova: use 'novaservice' service user rather than novaadmin

https://gerrit.wikimedia.org/r/1162063

gerritbot added a comment.Jun 20 2025, 8:04 PM

Change #1162077 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Openstack Nova: use 'novaservice' service user rather than novaadmin

https://gerrit.wikimedia.org/r/1162077

gerritbot added a comment.Jun 20 2025, 8:17 PM

Change #1162077 merged by Andrew Bogott:

[operations/puppet@production] Openstack Nova: use 'novaservice' service user rather than novaadmin

https://gerrit.wikimedia.org/r/1162077

Andrew added a subtask: T273150: OpenStack services should use system users to talk to Keystone.Jun 26 2025, 5:05 PM
taavi moved this task from Blocked to Inbox on the cloud-services-team board.Sep 24 2025, 2:17 PM
Andrew closed subtask T273150: OpenStack services should use system users to talk to Keystone as Resolved.Nov 14 2025, 7:18 PM
Andrew updated the task description. (Show Details)
Andrew closed this task as Resolved.Nov 14 2025, 8:21 PM

Keystone logs are still fairly full of warnings like

"DeprecationWarning: Policy enforcement is depending on the value of token."

however, if I delete /all/ keystone custom policies and let it just rely on code defaults, it still throws those warnings. So I guess they'll always be with us.

bd808 added a comment.Nov 14 2025, 10:20 PM
In T330759#11375496, @Andrew wrote:

Keystone logs are still fairly full of warnings like

"DeprecationWarning: Policy enforcement is depending on the value of token."

however, if I delete /all/ keystone custom policies and let it just rely on code defaults, it still throws those warnings. So I guess they'll always be with us.

The Keystone test suite silences these via warnings.filterwarnings: https://github.com/openstack/keystone/blob/70d34239e2cc86c363fb69808559bf7dea8f3432/keystone/tests/unit/ksfixtures/warnings.py#L39

The indirection of oslo.context makes it difficult to see exactly where this is actually being triggered in the runtime system: https://github.com/openstack/oslo.context/blob/0184e52950cf59e6300f28473eb956adbd43d266/oslo_context/context.py#L101-L106

The token key that is being complained about is setup in https://github.com/openstack/keystone/blob/70d34239e2cc86c363fb69808559bf7dea8f3432/keystone/common/context.py#L41-L66. The way this object works, anything that uses token in a policy lookup through a keystone.common.context.RequestContext object will trigger the warning.

Climbing out of this rabbit hole for now. We might be able to figure out more in a controlled runtime where we used PYTHONWARNINGS=error::DeprecationWarning in the environment to turn the warnings into exceptions so we could get the stacktrace.

Andrew added a comment.Nov 14 2025, 11:40 PM

I think I care about deprecation warnings when they apply to our custom policies, but don't care when keystone is issuing warnings about policies that shipped directly from keystone upstream. I'm happy assuming they're approximately a 'note to self' from the keystone team and ignoring them unless you think I'm missing something.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits