We should be able to run smoke tests (unit, API, Selenium...) during deployment. The question is how to do it.
#1 Smoke tests run from deployer's machine
- No development needed.
- Deployment documentation (example: SWAT deploys/Deployers) has an extra step where the deployer runs tests (example: npm run smoke) targeting a machine where the code is deployed (example: mwdebug1002)
- Each deployer has to set up machine so every type of smoke tests would run.
- A deployer might forget or decide not to run tests.
#2 Smoke tests run from scap
- It happens automatically.
- Deployers do not need any special setup on their machines.
- Requires some development.
#3 Something else?