(i just changed a typo in the task description)

I think we're going to need an A/B test before deploying automatic subscriptions as an opt-out/default-on feature. A typical process would look like:

• Offer it as a Beta Feature at a few wikis
• Get feedback/fix problems
• Run A/B test at multiple wikis
• See whether it works.
• Decide whether to deploy it widely.

In this case, I think the A/B test needs to include some "user sentiment" metrics. We don't have the tools that we really need to do this easily, because T89970: Enable microsurveys for long-term tracking of editing experience hasn't been built, but some of the pieces exist, and we might be able to manage.

IMO we need a solid baseline measurement plus to check back after a month or two, to see if people's views have changed. The main thing I need to know is whether automatic topic subscriptions tend to make editors angrier and/or more despairing about their fellow editors. If the views changed, during the trial, from "we mostly get along okay" to "people seem to be mad all the time", then we would have to conclude that automatic topic subscriptions are harmful and shouldn't be deployed widely.

Also, there isn't a clean division, in terms of the affective experience of the community between individuals who get the feature and individuals at the same wiki that don't. If those two people spent all day yelling at each other because one or both of them gets alerted the instant the other editor replies, then their public dispute affects everyone. We might need to run an A/B test that randomizes whole wikis to not get automatic topic subscriptions at all.

Because we need to collect non-automatic information, we might need to run the A/B test on larger wikis. (Problem: The feature's effect may not be the same on small wikis as it is on the largest wikis.)

Because we need to collect non-automatic information, we will need to get a privacy statement written by Legal, and that might affect which languages we choose. (On the Legal side, surveys are easier in some countries than in others, but I believe that we can choose most of them.)

Bringing this back to the main topic of this task: Before we decide where to offer it as the first Beta Feature, we probably need to decide if there are wikis that we want to "save" for the A/B test.

Peter and I are now talking about starting this as an opt-in-only feature, and for the next phase, as a double-opt-in feature. You would have to go to [[Special:Preferences#mw-prefsection-betafeatures]] to opt into "Discussion tools" and then also go to [[Special:Preferences#mw-prefsection-editing-discussion]] to turn on automatic topic subscriptions.

Because of this, IMO the first phase of offering this is low risk from the user perspective. We could offer this on any wiki or all of them.

However, we should probably first:

• Check with @MNeisler to make sure that the upcoming A/B test for the New Discussion tool will be using any retrospective data (e.g., comparing October 2021 vs November 2021) or otherwise affected by this. Automatic topic subscriptions could change the number and speed of replies posted, and even block rates (e.g., if an admin is more likely to notice a reply to a warning on a new user's talk page).

If the A/B test could be affected, then we should exclude those ~20 Wikipedias from the first deployments.

In T290502#7472886, @Whatamidoing-WMF wrote:

Peter and I are now talking about starting this as an opt-in-only feature, and for the next phase, as a double-opt-in feature. You would have to go to [[Special:Preferences#mw-prefsection-betafeatures]] to opt into "Discussion tools" and then also go to [[Special:Preferences#mw-prefsection-editing-discussion]] to turn on automatic topic subscriptions.

Because of this, IMO the first phase of offering this is low risk from the user perspective. We could offer this on any wiki or all of them.

However, we should probably first:

• Check with @MNeisler to make sure that the upcoming A/B test for the New Discussion tool will be using any retrospective data (e.g., comparing October 2021 vs November 2021) or otherwise affected by this. Automatic topic subscriptions could change the number and speed of replies posted, and even block rates (e.g., if an admin is more likely to notice a reply to a warning on a new user's talk page).

If the A/B test could be affected, then we should exclude those ~20 Wikipedias from the first deployments.

Great spot thinking ahead to see how Automatic Topic Subscriptions and the New Discussion Tool A/B test could impact one another, @Whatamidoing-WMF.

After talking with @MNeisler, we came to agree with what you proposed above: to exclude the ~20 Wikipedias participating in the New Discussion Tool A/B test (see T291306) from the initial Automatic Topic Subscription beta feature deployment (T282636).

To "codify" the above, I have:

1. Updated T290500 with the list of wikis where we would like the feature to initially be made available.
2. Filed T295424 to remind us to enable Automatic Topic Subscriptions at the ~20 Wikipedias participating in the New Discussion Tool A/B test once said A/B test concludes
3. Filed T295428 to make volunteers aware when Automatic Topic Subscriptions will become available as an opt-in beta feature