Page MenuHomePhabricator

New Impact Module: Start experiment for the new Impact module on Growth Pilot wikis (ar, bn, cs, es)
Closed, ResolvedPublic

Description

User stories:

  • As a newcomer, I want access to the new and improved Impact module, because seeing more personalized stats and visualization will help keep me engaged.
  • As an existing user with access to the Newcomer Homepage, I also want access to the new and improved Impact module, for the same reason as the newcomers.
  • As a Growth team member, I want to know what the effect of the new and improved Impact module is on the newcomer experience, which means we will run a controlled experiment with said module.

Notes

  • 50% of new user registrations are assigned to the new impact module, 50% will see the old module
  • All existing users will have the new impact module as the default
  • Add an Image experiment will be disabled as we move to this Impact Module A/B test (T323686)

Acceptance Criteria:

For users on ar, bn, cs, or es Wikipedia:

  • If I am an existing user with the Newcomer Homepage enabled I see the new Impact module when I visit the Homepage
  • If I register on the wiki during the experiment there is a 50% chance I am assigned to the new Impact module group and see that when I visit the Homepage, otherwise I see the old Impact module
  • If I register on the wiki during the experiment, my default tasks are the Add a Link structured task and the copy edit unstructured task, with the Add an Image task available for selection in the task menu (T323686)

Event Timeline

A/B test with a 50/50 split on the pilot wikis

Does this mean 50% of new registrations only? Or 50% of all existing users, plus 50% of new registrations, should have access to it?

kostajh renamed this task from New Impact Module: enable the new Impact module on Growth Pilot wikis (ar, bn, cs, es) to New Impact Module: Start experiment for the new Impact module on Growth Pilot wikis (ar, bn, cs, es).Nov 22 2022, 3:28 PM
kostajh updated the task description. (Show Details)

Change 859545 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[operations/mediawiki-config@master] GrowthExperiments: Allow accessing NewImpact module in production

https://gerrit.wikimedia.org/r/859545

@kostajh - Will users at pilot wikis have access to view the new impact module early with a URL parameter prior to the general rollout? (For example with a new-impact=1 parameter?)

If yes, what's the ETA so I can let Ambassadors know. Thanks.

@kostajh and @KStoller-WMF : I've updated the task description, hopefully making it clear what we're planning to do. I did add a specification of the default task types as part of ending the Add an Image experiment, with those defaults applying to all new registrations. Let me know if something's unclear or there are questions.

@kostajh and @KStoller-WMF : I've updated the task description, hopefully making it clear what we're planning to do. I did add a specification of the default task types as part of ending the Add an Image experiment, with those defaults applying to all new registrations. Let me know if something's unclear or there are questions.

LGTM, thanks!

@nettrom_WMF if it's OK with you, what I was thinking is:

  • control variant users all get the new impact module
  • users in the oldimpact variant *do not* get the new impact module

That is probably the easiest way to fit the experiment requirements (enable for all existing users, don't enable for 50% of new registrations).

Edit: changed the proposed experiment group name to oldimpact

Change 860866 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@master] NewImpact: Prepare experiment

https://gerrit.wikimedia.org/r/860866

Change 860867 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[operations/mediawiki-config@master] GrowthExperiments: Start newimpact experiment

https://gerrit.wikimedia.org/r/860867

Change 860866 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] NewImpact: Prepare experiment

https://gerrit.wikimedia.org/r/860866

Change 861816 had a related patch set uploaded (by Sergio Gimeno; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@wmf/1.40.0-wmf.10] NewImpact: Prepare experiment

https://gerrit.wikimedia.org/r/861816

Change 861816 abandoned by Sergio Gimeno:

[mediawiki/extensions/GrowthExperiments@wmf/1.40.0-wmf.10] NewImpact: Prepare experiment

Reason:

https://gerrit.wikimedia.org/r/861816

Change 861469 had a related patch set uploaded (by Sergio Gimeno; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@wmf/1.40.0-wmf.12] NewImpact: Prepare experiment

https://gerrit.wikimedia.org/r/861469

Change 861469 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@wmf/1.40.0-wmf.12] NewImpact: Prepare experiment

https://gerrit.wikimedia.org/r/861469

Change 861963 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/GrowthExperiments@master] User impact: Make the URL opt-in override the config flag

https://gerrit.wikimedia.org/r/861963

Change 861480 had a related patch set uploaded (by Kosta Harlan; author: Gergő Tisza):

[mediawiki/extensions/GrowthExperiments@wmf/1.40.0-wmf.12] User impact: Make the URL opt-in override the config flag

https://gerrit.wikimedia.org/r/861480

Change 861963 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] User impact: Make the URL opt-in override the config flag

https://gerrit.wikimedia.org/r/861963

Change 861480 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@wmf/1.40.0-wmf.12] User impact: Make the URL opt-in override the config flag

https://gerrit.wikimedia.org/r/861480

Mentioned in SAL (#wikimedia-operations) [2022-11-30T08:50:31Z] <urbanecm@deploy1002> Started scap: 5b2f3ae4db0e2d88c90ef00d48f0673dc9ff83b5: User impact: Make the URL opt-in override the config flag (T323526)

Mentioned in SAL (#wikimedia-operations) [2022-11-30T08:56:04Z] <urbanecm@deploy1002> Finished scap: 5b2f3ae4db0e2d88c90ef00d48f0673dc9ff83b5: User impact: Make the URL opt-in override the config flag (T323526) (duration: 05m 33s)

Change 862839 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[operations/mediawiki-config@master] GrowthExperiments: Enable new impact module on testwiki

https://gerrit.wikimedia.org/r/862839

Change 862839 merged by jenkins-bot:

[operations/mediawiki-config@master] GrowthExperiments: Enable new impact module on testwiki

https://gerrit.wikimedia.org/r/862839

Mentioned in SAL (#wikimedia-operations) [2022-12-01T14:39:15Z] <kharlan@deploy1002> Started scap: Backport for [[gerrit:862839|GrowthExperiments: Enable new impact module on testwiki (T323526)]]

Mentioned in SAL (#wikimedia-operations) [2022-12-01T14:40:23Z] <kharlan@deploy1002> kharlan and kharlan: Backport for [[gerrit:862839|GrowthExperiments: Enable new impact module on testwiki (T323526)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet

Mentioned in SAL (#wikimedia-operations) [2022-12-01T14:45:28Z] <kharlan@deploy1002> Finished scap: Backport for [[gerrit:862839|GrowthExperiments: Enable new impact module on testwiki (T323526)]] (duration: 06m 12s)

Change 859545 merged by Urbanecm:

[operations/mediawiki-config@master] GrowthExperiments: Remove unused GEAllowAccessToNewImpactModule

https://gerrit.wikimedia.org/r/859545

Change 860867 merged by jenkins-bot:

[operations/mediawiki-config@master] GrowthExperiments: Start oldimpact experiment

https://gerrit.wikimedia.org/r/860867

Mentioned in SAL (#wikimedia-operations) [2022-12-06T16:33:47Z] <kharlan@deploy1002> Started scap: Backport for [[gerrit:860867|GrowthExperiments: Start oldimpact experiment (T323526)]]

Mentioned in SAL (#wikimedia-operations) [2022-12-06T16:35:42Z] <kharlan@deploy1002> kharlan and kharlan: Backport for [[gerrit:860867|GrowthExperiments: Start oldimpact experiment (T323526)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet

Mentioned in SAL (#wikimedia-operations) [2022-12-06T16:44:42Z] <kharlan@deploy1002> Finished scap: Backport for [[gerrit:860867|GrowthExperiments: Start oldimpact experiment (T323526)]] (duration: 10m 54s)