In T404600, the Editing Team will be running a controlled experiment to evaluate the impact of the Suggestion Mode MVP.
This task involves the work of deciding whether this experiment will run on Test Kitchen.
Decision(s) to be made
- 1. Will the Suggestion Mode MVP controlled experiment (T404600) run on Test Kitchen?
Open questions
- 1. Does Test Kitchen support the metrics the Suggestion Mode MVP controlled experiment (T404600) specifies?
- Yes, per offline discussion @MNeisler.
- 2. Does Test Kitchen support multi-variate tests?
- Yes, per offline discussion with @JVanderhoop-WMF and we came to think that a multi-variate test is not needed for this experiment.
- 3. How (if at all) has the percentage of logged out traffic we can include in a controlled experiment increased from the current 0.1% threshold?
- See: T418530#11704018
Done
- Answers to all "Open questions" are documented
- Paths forward for all "Decision(s) to be made" are documented