Change Details

This task is about setting up the A/B test for the newcomer homepage. In order to understand the homepage’s impact on editor activation and retention, we propose an experiment that will be running for at least six months. During that time, 50% of new registrations on target wikis will have the homepage enabled by default, along with the rest of the homepage treatment, and 50% will have it disabled. These 50% pieces should be independent of splits made for any other experiment (such as for [[ https://phabricator.wikimedia.org/T206720 | help panel ]] or welcome survey). By “homepage treatment” we mean that a user both has the homepage on by default and is receiving the various ancillary features that drive them to the homepage, which may include banners, talk page messages, or links on Main Page. In the initial version, the only ancillary feature will be the fact that the username link in personal tools will go to the homepage. So in other words, when someone is in the treatment group, it may mean that they have several things turned on, and the control would have none of the things turned on. We may also want to run deeper splits later on, such as splitting the treatment group (who get the newcomer homepage treatment) into two groups -- one that has a certain homepage configuration or set of treatments and another that doesn't. When users are assigned to treatment and control groups, we will need to know which groups they are in, so they can be separated in the analysis. See the "Experiment plan" section in the [[ https://docs.google.com/document/d/1VzAq6VpOayvNe7V6_JzVEo2CRbv1SCj53ukeHLOhAeE/edit# | measurement specifications ]] for full details.

This task is about setting up the A/B test for the newcomer homepage. In order to understand the homepage’s impact on editor activation and retention, we propose an experiment that will be running for at least six months. During that time, 50% of new registrations on target wikis will have the homepage enabled by default, along with the rest of the homepage treatment, and 50% will have it disabled. These 50% pieces should be independent of splits made for any other experiment (such as for [[ https://phabricator.wikimedia.org/T206720 | help panel ]] or welcome survey). By “homepage treatment” we mean that a user both has the homepage on by default and is receiving the various ancillary features that drive them to the homepage, which may include banners, talk page messages, or links on Main Page. In the initial version, the only ancillary feature will be the fact that the username link in personal tools will go to the homepage. So in other words, when someone is in the treatment group, it may mean that they have several things turned on, and the control would have none of the things turned on. We may also want to run deeper splits later on, such as splitting the treatment group (who get the newcomer homepage treatment) into two groups -- one that has a certain homepage configuration or set of treatments and another that doesn't. When users are assigned to treatment and control groups, we will need to know which groups they are in, so they can be separated in the analysis. See the "Experiment plan" section in the [[ https://docs.google.com/document/d/1VzAq6VpOayvNe7V6_JzVEo2CRbv1SCj53ukeHLOhAeE/edit# | measurement specifications ]] for full details. When we analyze the experiment, we will want to look at these KPIs: * Activation * Retention * Edit count/volume * Productive edits (non-reverted) * Having a confirmed email address * Having created a user page (even if subsequently deleted)

This task is about setting up the A/B test for the newcomer homepage. In order to understand the homepage’s impact on editor activation and retention, we propose an experiment that will be running for at least six months. During that time, 50% of new registrations on target wikis will have the homepage enabled by default, along with the rest of the homepage treatment, and 50% will have it disabled. These 50% pieces should be independent of splits made for any other experiment (such as for [[ https://phabricator.wikimedia.org/T206720 | help panel ]] or welcome survey). By “homepage treatment” we mean that a user both has the homepage on by default and is receiving the various ancillary features that drive them to the homepage, which may include banners, talk page messages, or links on Main Page. In the initial version, the only ancillary feature will be the fact that the username link in personal tools will go to the homepage. So in other words, when someone is in the treatment group, it may mean that they have several things turned on, and the control would have none of the things turned on. We may also want to run deeper splits later on, such as splitting the treatment group (who get the newcomer homepage treatment) into two groups -- one that has a certain homepage configuration or set of treatments and another that doesn't. When users are assigned to treatment and control groups, we will need to know which groups they are in, so they can be separated in the analysis. See the "Experiment plan" section in the [[ https://docs.google.com/document/d/1VzAq6VpOayvNe7V6_JzVEo2CRbv1SCj53ukeHLOhAeE/edit# | measurement specifications ]] for full details. When we analyze the experiment, we will want to look at these KPIs: * Activation * Retention * Edit count/volume * Productive edits (non-reverted) * Having a confirmed email address * Having created a user page (even if subsequently deleted)