We want to use [[ https://wikitech.wikimedia.org/wiki/Test_Kitchen | Test Kitchen ]] for future A/B testing. Let's make an experimental change that is controlled through this framework, and learn about what is involved.
- [ ] Read technical documentation and examples, especially https://wikitech.wikimedia.org/wiki/Test_Kitchen/Conduct_an_experiment
- [ ] Try to record what we learn here in the task.
- [x] Write a docker-dev module to enable Test Kitchen (see https://wikitech.wikimedia.org/wiki/Test_Kitchen/Local_development_setup )
- Install and enable the MetricsPlatform extension
- Configure event streams
- Configure a local EventGate server
- [x] Code for review: https://gitlab.com/wmde/technical-wishes/docker-dev/-/merge_requests/188
- [x] Make a small but visible change to the reference dialog, for example changing the color of a button. {F71539189}
- [ ] Code for review: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Cite/+/1222589
- [x] Control this change through the test framework, using the [[ https://wikitech.wikimedia.org/wiki/Test_Kitchen/SDK/JavaScript_SDK | JavaScript SDK ]].
- [x] Set up instrumentation so that we know that people clicking the button were in condition A or B.
- [x] Set up event schema and streams; test both enrollment cases.
( We're not setting up a custom schema or stream for the test )
- [ ] Deploy the change to *only* test.wikipedia.org (~~using a feature flag or~~ the Test Kitchen experiment configuration).
- [ ] Generate some data and verify that we can find it.