Deployment-prep is thought as "staging" area to production, so it includes a lot of the production puppet roles.
This means that anyone willing to change something in puppet (even simple refactors) will be faced with the following issues:
- There is no greppable way to find out where that role is applied on within deployment-prep
- Hiera is distributed between on-disk hieradata, the horizon UI (in multiple places), and wikitech.
This results in a lot of time lost chasing things around all those places, and a general difficulty at finding out which classes are actually applied there, and how their data is organized (in a different model than in production).
So my proposal would be to actually make deployment-prep work as similarly to production as possible, while allowing changes to be made to deployment-prep to be made by non-opsens (e.g. people who don't have +2 on ops/puppet).
My idea is as follows (10k ft view):
- on the deployment-prep puppetmaster, configure a 'staging' environment for puppet, with its own site.pp that will include deployment-prep nodes declaration, instead of production ones. Those will come from a separate repository with its own merge/commit rights, separated from ops/puppet
- on the deployment-prep puppetmaster, define a disk-based hiera hierarchy to mimic 1:1 what we have in production, as far as the hierachy goes. This hiera hierarchy will also be in the new repository, thus freed from ops +2 need
- We might want to have all nodes in this environment derive from a base node that includes all the labs boilerplate that is needed for openstack/ldap/basic system.
This would allow me as an opsen working on continuous refactors of our production code to quickly know where to modify deployment-prep in order not to break it, and give greater logic and ordering to the hiera definitions for deployment-prep, which is now out of control, see:
- And of course the various prefixes in the Horizon UI
I think this would be a mid-level effort (with the largest task being reorganizing that unruly mess of hiera I just described) and real benefits both on the short and long term.