We have decided that we will not be implementing controlled A/B tests to measure the changes we will be making for this emailability initiative. That's because of the effort required around running experiments, because we intend to make the sort of changes that we don't expect to have negative impacts, and because increasing emailability is not a core goal of our team.
Instead, we want to do some basic longitudinal tracking. We want to know:
- Number of accounts created, by day, during the weeks surrounding the change.
- Percent of users adding emails on Special:CreateAccount during the few weeks leading up to the change.
- Percent of users adding emails on Special:CreateAccount during the few weeks after the change.
- Percent of users verifying their emails during the few weeks leading up to the change.
- Percent of users verifying their emails during the few weeks after to the change.
All of this should be split by platform (desktop/mobile) and wiki (Czech/Korean/Vietnamese).
We may also want to remove (or otherwise account for) users who add email addresses via the welcome survey or help panel features.