Page MenuHomePhabricator

Support OpenStack projects where project name != project id
Open, Needs TriagePublic

Description

Since the early days of OpenStack we've used a model where project ids and project names are identical. This was the norm in early OpenStack releases.

Modern openstack assigns arbitrary UUIDs to new projects, allowing project names to be changed on the fly without breaking db relationships. Our current deployments skip the UUID issue via a 'wmfkeystonehooks' hack that resets the project id to the project name.

UUIDs are annoying, but diverging from the upstream is also getting ever more annoying. If we adopt the usptream UUID practice, we'll have to locate and fix all of our scripts and workflows that rely on project id and name being the same.

The good news is we can change these workflows incrementally without actually breaking anything in the meantime. The breakage will only come later when we remove the keystone hack.

  • Designate domain assignment (e.g. myinstance.myproject.eqiad1.wikimedia.cloud) uses project id rather than project name
  • Cumin, spicerack, and cookbooks take project ids rather than project names. Either users will have to start using UUIDs or something in the cumin code will need to do a lookup and take distinct --project-id or project-name arguments
  • mwopenstackclients.py needs an audit to confirm that arguments are properly named 'project_name' or 'project_id' and used accordingly.

Event Timeline

Andrew updated the task description. (Show Details)

Change 955973 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] wmf_sink: don't assume project_name == project_id

https://gerrit.wikimedia.org/r/955973

Change 956088 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] nova_fullstack_test: don't assume project_name == project_id

https://gerrit.wikimedia.org/r/956088

Change 955973 merged by Andrew Bogott:

[operations/puppet@production] wmf_sink: don't assume project_name == project_id

https://gerrit.wikimedia.org/r/955973

Change 956088 merged by Andrew Bogott:

[operations/puppet@production] nova_fullstack_test: don't assume project_name == project_id

https://gerrit.wikimedia.org/r/956088

right now puppet cert names are the same as dns fqdns. It would be slightly less ambiguous for the puppet certname to include the project id instead... are there reasons why we depend on fqdn == cert name?

right now puppet cert names are the same as dns fqdns. It would be slightly less ambiguous for the puppet certname to include the project id instead... are there reasons why we depend on fqdn == cert name?

On further thought, I believe the puppet literally uses the fqdn as the cert name. So one way or another they need to stay the same.

right now puppet cert names are the same as dns fqdns. It would be slightly less ambiguous for the puppet certname to include the project id instead... are there reasons why we depend on fqdn == cert name?

@fnegri suggests that we use project IDs on the VM and puppet, and have designate-sink create cnames with the project name.

Change 956925 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] dynamicproxy: clarify that 'project name' was actually project_id all along

https://gerrit.wikimedia.org/r/956925

Change 956927 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] mwopenstackclients: add methods to correlate project id with name

https://gerrit.wikimedia.org/r/956927

Change 956928 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] wmcs-cold-migrate: remove instance_fqdn output hint

https://gerrit.wikimedia.org/r/956928

Change 956929 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] wmcs-instance-fqdns: support cases where project_name != project_id

https://gerrit.wikimedia.org/r/956929

Change 956930 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] wmcs-novastats-dnsleaks.py: Support project_id != project_name

https://gerrit.wikimedia.org/r/956930

Change 956927 merged by Andrew Bogott:

[operations/puppet@production] mwopenstackclients: add methods to correlate project id with name

https://gerrit.wikimedia.org/r/956927

Change 956928 merged by Andrew Bogott:

[operations/puppet@production] wmcs-cold-migrate: remove instance_fqdn output hint

https://gerrit.wikimedia.org/r/956928

Change 956929 abandoned by Andrew Bogott:

[operations/puppet@production] wmcs-instance-fqdns: support cases where project_name != project_id

Reason:

Going to stick with project_id in the fqdn

https://gerrit.wikimedia.org/r/956929

Change 957371 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] designate nova_fixed_multi: create A record using project_id and project_name

https://gerrit.wikimedia.org/r/957371

Change 956930 merged by Andrew Bogott:

[operations/puppet@production] wmcs-novastats-dnsleaks.py: Support project_id != project_name

https://gerrit.wikimedia.org/r/956930

Change 956925 merged by Majavah:

[operations/puppet@production] dynamicproxy: clarify that 'project name' was actually project_id all along

https://gerrit.wikimedia.org/r/956925

Change 957371 merged by Andrew Bogott:

[operations/puppet@production] designate nova_fixed_multi: create A recs using project_id and project_name

https://gerrit.wikimedia.org/r/957371

Change 988047 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[cloud/wmcs-cookbooks@main] enc.py: rename project_name arg to project_id

https://gerrit.wikimedia.org/r/988047

Change 988050 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] mwopenstackclients.py: remove a use of project_name

https://gerrit.wikimedia.org/r/988050

Change 988051 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] cloud-vps puppet encapi: use project_id instead of project_name for keystone

https://gerrit.wikimedia.org/r/988051

Change 988052 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Keystone: remove hack ensuring that project_id == project_name

https://gerrit.wikimedia.org/r/988052

Change 988050 merged by Andrew Bogott:

[operations/puppet@production] mwopenstackclients.py: remove a use of project_name

https://gerrit.wikimedia.org/r/988050

Change 988051 merged by Andrew Bogott:

[operations/puppet@production] cloud-vps puppet encapi: use project_id instead of project_name for keystone

https://gerrit.wikimedia.org/r/988051

Change 988047 merged by jenkins-bot:

[cloud/wmcs-cookbooks@main] enc.py: rename project_name arg to project_id

https://gerrit.wikimedia.org/r/988047