Change Details

This task is about publishing the rationale for the A/B test on the Reply Tool project page on mw.org. [i] ===Update components - What questions we are trying to use this A/B test to answer - What metrics we will use to compare the two A/B test's two test groups - What decision(s) the results of this A/B test will help us make === Done - [ ] An update is published to the Reply Tool project page [i] that contains the `===Update components` listed above --- i. https://www.mediawiki.org/wiki/Talk_pages_project/replying

This task is about publishing the rationale for the A/B test on the Reply Tool project page on mw.org. [i] ===Update components - What questions we are trying to use this A/B test to answer - What metrics we will use to compare the two A/B test's two test groups - What decision(s) the results of this A/B test will help us make === Update language In the broadest of terms, this A/B test is a "temperature check"...a way for us to check whether the Reply Tool has caused any observable changes in who is using talk pages and how they are using them. Details below... **Decision to be made** This A/B test is intended to help us decide whether the Reply Tool should be made available to all people, at all wikis, as an opt-out user preference. **Decision inputs** To make this decision, we are interested in learning whether the Reply Tool is causing notable negative changes in the rates at which people start and continue participating on talk pages as well as the rates at which people are participating on talk pages in disruptive ways. The metrics we will use to evaluate "participation" and "disruption" are listed below. //See: "Test metrics."// Before that, it is important to note that we **do not consider** notable positive changes in the rates at which people start and continue participating on talk pages as well as the rates at which people are participating on talk pages in disruptive ways as prerequisites for the Reply Tool being made available to all people, at all wikis, as an opt-out user preference. We've come to think this for the following reasons: - Evidence to date suggests [ii] the Reply Tool is having the intended short-term impact: Junior and Senior Contributors have an easier time participating in existing conversations, "correctly." - We assume notable changes in the rates at which people start and continue participating on talk pages will likely get expressed over a longer period of time (read: they will not be detectable during the course of this A/B test). **Test metrics** //Please see T252057.// === Done - [ ] An update is published to the Reply Tool project page [i] that contains the `===Update components` listed above --- i. https://www.mediawiki.org/wiki/Talk_pages_project/replying ii. https://www.mediawiki.org/wiki/Talk_pages_project/replying#Metrics

This task is about publishing the rationale for the A/B test on the Reply Tool project page on mw.org. [i] ===Update components - What questions we are trying to use this A/B test to answer - What metrics we will use to compare the two A/B test's two test groups - What decision(s) the results of this A/B test will help us make === Update language In the broadest of terms, this A/B test is a "temperature check"...a way for us to check whether the Reply Tool has caused any observable changes in who is using talk pages and how they are using them. Details below... **Decision to be made** This A/B test is intended to help us decide whether the Reply Tool should be made available to all people, at all wikis, as an opt-out user preference. **Decision inputs** To make this decision, we are interested in learning whether the Reply Tool is causing notable negative changes in the rates at which people start and continue participating on talk pages as well as the rates at which people are participating on talk pages in disruptive ways. The metrics we will use to evaluate "participation" and "disruption" are listed below. //See: "Test metrics."// Before that, it is important to note that we **do not consider** notable positive changes in the rates at which people start and continue participating on talk pages as well as the rates at which people are participating on talk pages in disruptive ways as prerequisites for the Reply Tool being made available to all people, at all wikis, as an opt-out user preference. We've come to think this for the following reasons: - Evidence to date suggests [ii] the Reply Tool is having the intended short-term impact: Junior and Senior Contributors have an easier time participating in existing conversations, "correctly." - We assume notable changes in the rates at which people start and continue participating on talk pages will likely get expressed over a longer period of time (read: they will not be detectable during the course of this A/B test). **Test metrics** //Please see T252057.// === Done - [ ] An update is published to the Reply Tool project page [i] that contains the `===Update components` listed above --- i. https://www.mediawiki.org/wiki/Talk_pages_project/replying ii. https://www.mediawiki.org/wiki/Talk_pages_project/replying#Metrics