I like option 1 in theory, but in practice I think option 2 is the best choice at the moment.

Implementing option 1 well requires some work, we can reconsider that in the future.

I would like to see the 'TBD' parts filled up before expressing an opinion. Some ideas to fill them up:

I think it would be useful to try to define what are the user flows and audiences that we foresee will be served with this, pseudo-random example:

Actors:

• wmcs sres (high infra knowledge)
• clinic duty assignee (little infra knowledge)
• cloudvps user (little infra knowledge)

Flows:

• wmcs sres creating a new infra project (ex. metricsinfra or admin-monitoring)
• current clinic duty creating a new project for a user (ex. tools)
• current clinic duty changing a project member list
• cloudvps user self-creating a project

...

From there it will be easier to define the interfaces and the shape they should have to serve the audiences they are built for.

There are certain conflicts - or role overlap - between automation via cookbooks and automation via opentofu

I don't see this as a conflict, as long as the flows are defined it's equivalent imo (ex. if you are doing clinic duty and have to create a project -> cookbook, if you are wmcs sre and have to create an admin project -> tofu directly).

Tracking hundred of resources (the project resources themselves) may make the repository a bit more complex to maintain

This brings the question if tofu is actually making things simpler/easier for us or not overall. If it's better than not using it only for a very small amount of resources it might not be a good idea long-term. From my point of view it should be better than only cookbooks for managing many resources, and more than able to scale up, but please elaborate with your experience and the issues you have found so far.

In option 1, I do see the benefit of having the revisions tracking the project creation, even if the patches are done automatically from a cookbook (or even merged automatically!).

Another benefit of option 1 is consolidating the project creation flow, having projects created only by tofu interacting with openstack (as opposed to sometimes interacting directly, sometimes with tofu) allows for better checks, standardization, debugging, etc.

In T385604#10567062, @dcaro wrote:

I would like to see the 'TBD' parts filled up before expressing an opinion. Some ideas to fill them up:

I think it would be useful to try to define what are the user flows and audiences that we foresee will be served with this, pseudo-random example:
[...]

Continuing with this example, I see:

• actors: WMCS SREs
• flows: WMCS SREs creating a new infra project (ex. metricsinfra or admin-monitoring)

The other examples:

• current clinic duty changing a project member list --> We already decided not to track project membership via tofu-infra. Given project member list is self-service, it does not make sense to track in version control. Users can change this at any time.
• cloudvps user self-creating a project --> Not supported today. It would be nice to have though, but something based on opentofu is maybe not the right implementation for this, as it would be self-service. Similar to the previous point.
• current clinic duty creating a new project for a user (ex. tools) --> I'm leaning towards not having this in tofu-infra, i.e, option 2.

I propose we move forward with option 2.

Option 2 seems right to me!

It seems this is waiting for my vote, so though I still consider the pros and cons of both options incomplete, I lean for option 1.

I find having a single system for project creation worth it even if it means creating automatic patches. It also allows to attach metadata to the project creation (ex. the task, rationale, etc.) both in the commit and in a comment/readme file, etc. that otherwise is hard to keep track of (project description? openstack logs?)
That in my mind is one of the greatest advantages of using git to track infrastructure.

I think tofu should be able to scale to the amount of projects we have without many issues, if it does not, then maybe it's not the right tool to manage our setup. Having 500 directories/files/entries is not overwhelming imo (they are quite clearly named, and self-contained, thanks to the refactor).

And looking forward to managing quotas and other user-project resources, it makes even more sense to me to use the same methods to deploy them (tofu being the one that creates the resources), and the same entry point (cookbook being the one that uses tofu). I don't see tofu being a blocker for self-service in any way, just a building block/tool for it.

Said that, if the issue is effort to implement, then sure, if we don't have the capacity let's not do it, though that is not in the rationales for any of the options, so I assume it's not an issue.

Having reviewed and reconsidered this, I now vote for a modified version of option 1:

"Track all project definitions /in the default domain/ via tofu-infra."

I think that gets tofu out of the way of projects that are auto-generated by e.g. magnum which was my primary concern with managing all tenants.

In T385604#11299337, @Andrew wrote:

Having reviewed and reconsidered this, I now vote for a modified version of option 1:

"Track all project definitions /in the default domain/ via tofu-infra."

I think that gets tofu out of the way of projects that are auto-generated by e.g. magnum which was my primary concern with managing all tenants.

Agree with you, I was not taking into account this, they are probably better left on their own.

I modified the description to clarify that Option 1 would only apply to projects in the default domain, as I think that was always the intent although it was not clearly specified.

I also added Option 3 with another path that I would like to explore. Worth noting that choosing Option 1 would not preclude us from considering Option 3 later on.

I vote for the Option 1 (with Andrew's note on only for non-automatic projects), though Option 3 would be a close second (I don't completely understand the flows, though look interesting).

I'm fine with going with Option 1 if there is consensus, we can re-evaluate option 3 in the future.

Seems like consensus around option 1 -- let's close this next week if no one objects.