RESTBase is completely driven by its Swagger spec based configuration. This means that we often have a need to co-deploy the configuration and code in a coordinated manner. This is true not only for production, but also for testing.
The current process for coordinating config and code changes is to
- disable puppet on all nodes
- <s>beg</s> gently ask an ops person to merge the relevant puppet config
- manually re-enable puppet on one node
- deploy (using Ansible) to that one node only
- if successful, proceed with the next node
This is fairly complex, time intensive and easy to get wrong. Things going wrong has led to outages in the past.
Instead, we could move the RESTBase configuration into the RESTBase deploy system. This is mostly straightforward:
- config variables can be copied to group_vars
- templates can be converted to Jinja2 templates used by Ansible
- a stanza to deploy the config is added to the deploy task
However, we don't want to expose secrets like C* passwords in our deploy repo. Possible ways to still support this:
- symlink a private group_vars file in place on a deploy host
- submodule pulling in a private gerrit repository
- manual text file
- export private data from puppet on the destination system (to a file), and pull it into ansible at runtime
To me the combination of 3) for testing and 4) for production sounds most promising.
I have two main questions:
- do you think this is worth doing, and
- how do you think we should deal with the secret issue?