Change Details

We want to use [[ https://wikitech.wikimedia.org/wiki/Test_Kitchen | Test Kitchen ]] for future A/B testing. Let's make an experimental change that is controlled through this framework, and learn about what is involved. - [x] Read technical documentation and examples, especially https://wikitech.wikimedia.org/wiki/Test_Kitchen/Conduct_an_experiment - [x] Try to record what we learn here in the task. --> see below - [x] Write a docker-dev module to enable Test Kitchen (see https://wikitech.wikimedia.org/wiki/Test_Kitchen/Local_development_setup ) - Install and enable the MetricsPlatform extension - Configure event streams - Configure a local EventGate server - [x] Code for review: https://gitlab.com/wmde/technical-wishes/docker-dev/-/merge_requests/188 - [x] Make a small but visible change to the reference dialog, for example changing the color of a button. {F71539189} https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Cite/+/1222589 - [x] Control this change through the test framework, using the [[ https://wikitech.wikimedia.org/wiki/Test_Kitchen/SDK/JavaScript_SDK | JavaScript SDK ]]. - [x] Set up instrumentation so that we know that people clicking the button were in condition A or B. - [x] Set up event schema and streams; test both enrollment cases. ( We're not setting up a custom schema or stream for the test ) - [ ] Deploy the change to *only* test.wikipedia.org (~~using a feature flag or~~ the Test Kitchen experiment configuration). - [ ] Generate some data and verify that we can find it. ##For review Our learnings: [[ https://docs.google.com/document/d/1oVAEhBUaCS1plJuIXU6OAZSjfnnOoz4d-M1sZlMBr5Y/edit?usp=sharing | Test Kitchen Summary ]] Test code change on beta: http://dev.wiki.local.wmftest.net:8080/w/index.php?title=Example/Cite&veaction=edit&mpo=test-context-edit-color:green-button To test https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Cite/+/1222589 locally: # Start docker with eventlogging # Add `?mpo=test-context-edit-color:green-button` to the url. # Open a context item. # Enjoy the light green icon. :)

We want to use [[ https://wikitech.wikimedia.org/wiki/Test_Kitchen | Test Kitchen ]] for future A/B testing. Let's make an experimental change that is controlled through this framework, and learn about what is involved. - [x] Read technical documentation and examples, especially https://wikitech.wikimedia.org/wiki/Test_Kitchen/Conduct_an_experiment - [x] Try to record what we learn here in the task. --> see below - [x] Write a docker-dev module to enable Test Kitchen (see https://wikitech.wikimedia.org/wiki/Test_Kitchen/Local_development_setup ) - Install and enable the MetricsPlatform extension - Configure event streams - Configure a local EventGate server - [x] Code for review: https://gitlab.com/wmde/technical-wishes/docker-dev/-/merge_requests/188 - [x] Make a small but visible change to the reference dialog, for example changing the color of a button. {F71539189} https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Cite/+/1222589 - [x] Control this change through the test framework, using the [[ https://wikitech.wikimedia.org/wiki/Test_Kitchen/SDK/JavaScript_SDK | JavaScript SDK ]]. - [x] Set up instrumentation so that we know that people clicking the button were in condition A or B. - [x] Set up event schema and streams; test both enrollment cases. ( We're not setting up a custom schema or stream for the test ) - [x] Deploy the change https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Cite/+/1222589 - [x] Create AB Test for *only* test.wikipedia.org https://test-kitchen.wikimedia.org/experiment/test-context-edit-color - [ ] Generate some data and verify that we can find it. ##For review Our learnings: [[ https://docs.google.com/document/d/1oVAEhBUaCS1plJuIXU6OAZSjfnnOoz4d-M1sZlMBr5Y/edit?usp=sharing | Test Kitchen Summary ]] Test code change on beta: http://dev.wiki.local.wmftest.net:8080/w/index.php?title=Example/Cite&veaction=edit&mpo=test-context-edit-color:green-button To test https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Cite/+/1222589 locally: # Start docker with eventlogging # Add `?mpo=test-context-edit-color:green-button` to the url. # Open a context item. # Enjoy the light green icon. :)

We want to use [[ https://wikitech.wikimedia.org/wiki/Test_Kitchen | Test Kitchen ]] for future A/B testing. Let's make an experimental change that is controlled through this framework, and learn about what is involved. - [x] Read technical documentation and examples, especially https://wikitech.wikimedia.org/wiki/Test_Kitchen/Conduct_an_experiment - [x] Try to record what we learn here in the task. --> see below - [x] Write a docker-dev module to enable Test Kitchen (see https://wikitech.wikimedia.org/wiki/Test_Kitchen/Local_development_setup ) - Install and enable the MetricsPlatform extension - Configure event streams - Configure a local EventGate server - [x] Code for review: https://gitlab.com/wmde/technical-wishes/docker-dev/-/merge_requests/188 - [x] Make a small but visible change to the reference dialog, for example changing the color of a button. {F71539189} https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Cite/+/1222589 - [x] Control this change through the test framework, using the [[ https://wikitech.wikimedia.org/wiki/Test_Kitchen/SDK/JavaScript_SDK | JavaScript SDK ]]. - [x] Set up instrumentation so that we know that people clicking the button were in condition A or B. - [x] Set up event schema and streams; test both enrollment cases. ( We're not setting up a custom schema or stream for the test ) - [ - [x] Deploy the change toge https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Cite/+/1222589 - [x] Create AB Test for *only* test.wikipedia.org (~~using a feature flag or~~ the Test Kitchen https://test-kitchen.wikimedia.org/experiment configuration).t/test-context-edit-color - [ ] Generate some data and verify that we can find it. ##For review Our learnings: [[ https://docs.google.com/document/d/1oVAEhBUaCS1plJuIXU6OAZSjfnnOoz4d-M1sZlMBr5Y/edit?usp=sharing | Test Kitchen Summary ]] Test code change on beta: http://dev.wiki.local.wmftest.net:8080/w/index.php?title=Example/Cite&veaction=edit&mpo=test-context-edit-color:green-button To test https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Cite/+/1222589 locally: # Start docker with eventlogging # Add `?mpo=test-context-edit-color:green-button` to the url. # Open a context item. # Enjoy the light green icon. :)