An IDM is the kind of setup we cannot easily test in Cloud VPS since some of the workflows are tied to other production setups. We can however mimic setups which face the same issues, namely the IDPs and Netbox, which both have staging host(s) within production.
We could have an additional DNS name idm-next.w.o which points to an additional VM running the same Puppet setup (and the same deb), e.g. running on idm-test1001.wikimedia.org. This would allow us to:
- Deploy a more recent version of the deb first on the idm-next instance
- Apply idm-test-specific Hiera settings first to the test instance (which would e.g. enable a new module/feature which isn’t live for the main test instance)
To prevent users from accidentally using the -next version we could implement a filter which e.g. validates that the user logging in is part of the cn=ops or cn=idm-testers LDAP groups.