Session
- Track: Deploying and Hosting
- Topic: Quo Vadis Beta Cluster? Towards better testing and staging environments
Description
So called "beta" sites have been commonly used as testing and staging environment for code changes before these are shipped to production. There are several issues with these sites used like a pre-production environment
- they are hosted on the de facto not maintained infrastructure
- There is no easy automated way to set the configuration (in the sense of extensions and services enabled, features set enabled, and other config options) of the testing/staging site, not to mention being able to launch a pre-production environment with the particular configuration and software versions, e.g. when intending to test compatibility of the new feature with different versions of extensions, skins, etc.
- They are actual "permanent" sites, i.e. they're not suited for use as mid-/short-term living staging environments
In this session we will look into what requirements we would have for the staging environments used in our work, and also see what possible solutions could we see that could replace "beta" sites as testing and staging sites (e.g. using new possibilities allowed with the adoption of container-based solutions).
Questions to answer and discuss
Question: Why and when is parity with production important?
Significance: Understanding/level setting on why will help tease out requirements.
Question: What are the most impactful problems wrt testing environments today?
Significance: Understanding the challenges with past efforts will help inform future approaches.
Question: What are some high level ideas to improve testing environments?
Significance: As these set of problems will not be addressed in an hour session, having some seed ideas for future discussions would be helpful. We are all engineers that like to solve problems, not like we'll avoid diving into solutions anyways :-)
Related Issues
- Testing Pyramid - would you be ready to use these environments today?
- ...
Pre-reading for all Participants
- [add links here]
Notes document(s)
https://etherpad.wikimedia.org/p/WMTC19-T234643
Notes and Facilitation guidance
https://www.mediawiki.org/wiki/Wikimedia_Technical_Conference/2019/NotesandFacilitation
Session Leader(s)
Session Scribes
Session Facilitator
Session Style / Format
- [what type of format will this session be?]
Post-event summary:
- Test Environments are hard!
- When discussing "production-like", production data is often a strong desire. This could be scale of data as well, and most likely, diversity of data.
- Having multiple environments is likely part of the solution,
- There are many use cases/requirements from many different personas
- Software Engineering, Test Engineers, SREs, Product Managers
- Test activities are varied and have separate and sometimes conflicting requirements.
- Further requirements analysis is needed.
Post-event action items:
- Formation of the Test Environments WG
- Collaboration with Local Dev Environments work