Page MenuHomePhabricator

New Impact Module: Start experiment for the new Impact module on Growth Pilot wikis (ar, bn, cs, es)
Open, MediumPublic

Description

User stories:

  • As a newcomer, I want access to the new and improved Impact module, because seeing more personalized stats and visualization will help keep me engaged.
  • As an existing user with access to the Newcomer Homepage, I also want access to the new and improved Impact module, for the same reason as the newcomers.
  • As a Growth team member, I want to know what the effect of the new and improved Impact module is on the newcomer experience, which means we will run a controlled experiment with said module.

Notes

  • 50% of new user registrations are assigned to the new impact module, 50% will see the old module
  • All existing users will have the new impact module as the default
  • Add an Image experiment will be disabled as we move to this Impact Module A/B test (T323686)

Acceptance Criteria:

For users on ar, bn, cs, or es Wikipedia:

  • If I am an existing user with the Newcomer Homepage enabled I see the new Impact module when I visit the Homepage
  • If I register on the wiki during the experiment there is a 50% chance I am assigned to the new Impact module group and see that when I visit the Homepage, otherwise I see the old Impact module
  • If I register on the wiki during the experiment, my default tasks are the Add a Link structured task and the copy edit unstructured task, with the Add an Image task available for selection in the task menu (T323686)

Event Timeline

A/B test with a 50/50 split on the pilot wikis

Does this mean 50% of new registrations only? Or 50% of all existing users, plus 50% of new registrations, should have access to it?

kostajh renamed this task from New Impact Module: enable the new Impact module on Growth Pilot wikis (ar, bn, cs, es) to New Impact Module: Start experiment for the new Impact module on Growth Pilot wikis (ar, bn, cs, es).Tue, Nov 22, 3:28 PM
kostajh updated the task description. (Show Details)

Change 859545 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[operations/mediawiki-config@master] GrowthExperiments: Allow accessing NewImpact module in production

https://gerrit.wikimedia.org/r/859545

Previous patch for ending link recommendation experiment.

@kostajh - Will users at pilot wikis have access to view the new impact module early with a URL parameter prior to the general rollout? (For example with a new-impact=1 parameter?)

If yes, what's the ETA so I can let Ambassadors know. Thanks.

@kostajh and @KStoller-WMF : I've updated the task description, hopefully making it clear what we're planning to do. I did add a specification of the default task types as part of ending the Add an Image experiment, with those defaults applying to all new registrations. Let me know if something's unclear or there are questions.

@kostajh and @KStoller-WMF : I've updated the task description, hopefully making it clear what we're planning to do. I did add a specification of the default task types as part of ending the Add an Image experiment, with those defaults applying to all new registrations. Let me know if something's unclear or there are questions.

LGTM, thanks!

@nettrom_WMF if it's OK with you, what I was thinking is:

  • control variant users all get the new impact module
  • users in the oldimpact variant *do not* get the new impact module

That is probably the easiest way to fit the experiment requirements (enable for all existing users, don't enable for 50% of new registrations).

Edit: changed the proposed experiment group name to oldimpact

Change 860866 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@master] NewImpact: Prepare experiment

https://gerrit.wikimedia.org/r/860866

Change 860867 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[operations/mediawiki-config@master] GrowthExperiments: Start newimpact experiment

https://gerrit.wikimedia.org/r/860867