Deploying service charts using helmfile involves a step where the service is deployed to the staging cluster and tested. Instead of performing tests manually by running curl commands, services should provide a way to run a suite of end-to-end tests that ensure the service is functioning as intended an can be deployed to production.

Ideally, services would share a test framework and expose the test suite in a uniform way. Also, it should be easy to run the tests locally in a development environment (e.g. on minikube).

Proposal:

• Create a framework based on python's unittest package, aka pyunit. This is available on deployment hosts.
• Use make as a runner to provide a uniform way of invoking all kinds of tests and for additional setup steps,
• Place a makefile in a tests foler in the service directory and define a check target that invokes the tests.
• Allow the makefile to function in at least two environments, "staging" (the default) and "minikube" (for local testing)
• Define "minikube" as an environment in helmfile.yaml as well, wiuth a single deployment called "local".
• Place any test files in the tests directory. Normally these would be pyunit tests.
• Place reusable make snippets in a makefiles directory at the top level of the repo.
• Place reusable python packages in a pathon directory at the top level of the repo.
• Create a python module that can read helm value files, so tests can be written to check he bhavios based on what is in the value files.
• It should be possible to define local overrides in alue files anding in .local.yaml.