At present we do not detect hiera yaml syntax errors or data type errors in CI or prior to patch submission. This gap has been a long source of frustration for users. Recently @brouberol ran into this issue and proposed this [[ https://gerrit.wikimedia.org/r/c/operations/puppet/+/979119 | patch ]], which enables running our rspec compile test against our hiera data. The purpose of this task is to decide on our short term and long term approach to closing this gap and reducing this source of errors.
Possible routes:
# Run compile tests against our hiera data, [[ https://gerrit.wikimedia.org/r/c/operations/puppet/+/979119 | patch ]]
## Ensures our hiera data is consumable by our puppet code
## Ties our module specs to data outside of the module
# Run compile tests against in module data, i.e. move our data into the module, [[ https://gerrit.wikimedia.org/r/c/operations/puppet/+/979470 | patch ]]
## Ensures our hiera data is consumable by our puppet code
## Places our specific data in modules, making them less generic
# Generate JSON schemas based on our Puppet types and validate our hiera data using those schemas, [[ https://gerrit.wikimedia.org/r/c/operations/puppet/+/979469 | patch ]]
## Ensures our hiera data is consumable by our puppet code
## Allows us to keep our specific hiera data out of our modules
## At present creating the JSON schema is done manually, but it appears possible to generate the schemas programatically from the Puppet types