Page MenuHomePhabricator

Account creation: URL parameters
Closed, ResolvedPublic

Description

Donors will click on the call-to-action in their email and be brought to the wiki to create an account, hopefully to go on to edit. We want to be able to analyze the behavior of this cohort in all the ways that we usually analyze Growth newcomers through EventLogging and edit tags -- but we want to be able to separate out this cohort to see them distinctly.

We'll do this through identifying a URL query parameter in the call-to-action link. We may want to keep track of multiple query parameters if there are different subgroups in the cohort (e.g. new donors / repeat donors) or if we try different versions of the email. For example, the call to action link might be to https://es.wikipedia.org/wiki/Special:CreateAccount?campaign=growth-mexico-2021-a for variant A of an email or https://es.wikipedia.org/wiki/Special:CreateAccount?campaign=growth-mexico-2021-b for variant B, for an email campaign sent to donors in Mexico.

Engineering tasks

  • Hook into account creation process (like with the geForceVariant parameter) to look for the campaign parameter and then a known string pattern for the campaigns we want to track (e.g. growth-advancement-{blah-blah}
  • Create a hidden preference to store campaign URL query parameter for the user, for analytics purposes
  • Opt-in the user to Growth features (see also T284119: Growth: options for disabling/enabling Growth features for groups of users)

We have decided that there will only be two variants of the email: one for Portuguese-speaking recipients and one for Spanish-speaking recipients. These are the URLs they will use:

Details

SubjectRepoBranchLines +/-
mediawiki/extensions/GrowthExperimentswmf/1.37.0-wmf.12+19 -11
mediawiki/extensions/GrowthExperimentswmf/1.37.0-wmf.11+19 -6
mediawiki/extensions/GrowthExperimentsmaster+19 -11
mediawiki/extensions/GrowthExperimentswmf/1.37.0-wmf.11+58 -5
mediawiki/extensions/GrowthExperimentsmaster+58 -5
operations/mediawiki-configmaster+3 -3
mediawiki/extensions/GrowthExperimentswmf/1.37.0-wmf.9+4 -4
mediawiki/extensions/GrowthExperimentswmf/1.37.0-wmf.9+23 -7
mediawiki/extensions/GrowthExperimentswmf/1.37.0-wmf.9+279 -3
mediawiki/extensions/GrowthExperimentswmf/1.37.0-wmf.9+10 -302
mediawiki/extensions/GrowthExperimentswmf/1.37.0-wmf.9+302 -10
mediawiki/extensions/GrowthExperimentswmf/1.37.0-wmf.11+302 -10
mediawiki/extensions/GrowthExperimentsmaster+302 -10
mediawiki/extensions/GrowthExperimentsmaster+56 -6
mediawiki/extensions/GrowthExperimentsmaster+5 -1
Show related patches Customize query in gerrit

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

@kostajh -- could you specify in the task description what we'll need to know on the Growth team side in order to do this? So that Fundraising knows what is needed on their end. Thank you!

@nettrom_WMF -- I don't think you've heard much about these plans yet, but we're trying to move quickly on an idea with Fundraising. In short, we'll be emailing donors to ask them to create accounts. Then they'll create accounts, get picked up by Growth features, and hopefully go on to make their first edits. We need to be able to identify which accounts came through this workflow so that we can do our usual analytics, but restricted to just those accounts.

@kostajh proposes that we identify them using query parameters in the link they'll click in the email, followed by storing their status as a hidden preference. We would also use this query parameter to keep these participants out of our control groups, so that they all get the features.

As you think about this, what comes to mind? Do you foresee being able to easily work with these users via the hidden preference? Any concerns about keeping them out of control groups?

@MMiller_WMF : I set up a time for us to meet early next week to chat about this project so I don't miss anything important.

With regards to the idea to use the campaign query parameter at account creation and a hidden preference, that should work well to enable us to identify these accounts as it would mean that we can identify them when querying the Data Lake as well as the replicated databases.

I don't have immediate concerns about not having these accounts be randomly assigned to groups or not. The key parts to that are that 1) we document it so that we can exclude these from analysis where random assignment is expected, and 2) we'll be focusing on measurements related to conversion/participation to get a sense of success, rather than comparisons against a baseline (which a control group would've been).

Will being able to ensure that they're not in a control group mean that we're tying T284119 to this?

Lastly, if these users will see the Welcome Survey after account creation, the first question asks why they created an account today. None of the options appear to really fit this case, unless we're expecting them to have a second motivation?

I'll look a little more closely at this next time I have some time to spare for it, those where the things I could identify so far.

kostajh updated the task description. (Show Details)
kostajh edited subscribers, added: mewoph, DMburugu, Tgr; removed: nettrom_WMF, MeganHernandez_WMF, spatton and 5 others.

Change 700670 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@master] Make it possible to opt-out of Growth features

https://gerrit.wikimedia.org/r/700670

Change 700671 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@master] Make it possible for users to opt-out of Growth features

https://gerrit.wikimedia.org/r/700671

@nettrom_WMF and I talked about this today. We agreed that the plan laid out here (query parameter saved as hidden preference) is going to work for our purposes. We also reminded ourselves that these users will not be getting the welcome survey; we're going to send them straight from account creation to their homepage. Therefore, we don't want to record anything for their welcome survey data -- we should leave that blank.

Change 701097 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@master] Donor campaign: use hidden preference to store campaign

https://gerrit.wikimedia.org/r/701097

Change 701106 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/GrowthExperiments@master] Add custom signup flow for donors

https://gerrit.wikimedia.org/r/701106

Change 701097 abandoned by Gergő Tisza:

[mediawiki/extensions/GrowthExperiments@master] Donor campaign: use hidden preference to store campaign

Reason:

Squashed into Icfff8893f08d5ad1c161472210ad2707846074a8 for easier backporting.

https://gerrit.wikimedia.org/r/701097

Change 700670 abandoned by Urbanecm:

[mediawiki/extensions/GrowthExperiments@master] Make it possible to opt-out of Growth features

Reason:

Per discussion, let's go with I08ab0e4f9d instead.

https://gerrit.wikimedia.org/r/700670

Change 701106 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Add custom signup flow for donors

https://gerrit.wikimedia.org/r/701106

Change 700741 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.9] Add custom signup flow for donors

https://gerrit.wikimedia.org/r/700741

Change 700742 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.11] Add custom signup flow for donors

https://gerrit.wikimedia.org/r/700742

Change 700741 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.9] Add custom signup flow for donors

https://gerrit.wikimedia.org/r/700741

Change 700742 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.11] Add custom signup flow for donors

https://gerrit.wikimedia.org/r/700742

Mentioned in SAL (#wikimedia-operations) [2021-06-23T18:53:33Z] <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.11/extensions/GrowthExperiments/: 76e5fc91083736d14049a05ed227cdea015c113e: Add custom signup flow for donors (T284740; T284800; T285281) (duration: 01m 07s)

Mentioned in SAL (#wikimedia-operations) [2021-06-23T18:55:42Z] <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.9/extensions/GrowthExperiments/: REVERT: 76e5fc91083736d14049a05ed227cdea015c113e: Add custom signup flow for donors (T284740; T284800; T285281) (duration: 00m 38s)

Change 700744 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.9] Revert "Add custom signup flow for donors"

https://gerrit.wikimedia.org/r/700744

Change 700744 merged by Urbanecm:

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.9] Revert "Add custom signup flow for donors"

https://gerrit.wikimedia.org/r/700744

Mentioned in SAL (#wikimedia-operations) [2021-06-23T19:07:32Z] <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.9/extensions/GrowthExperiments/: 2338e53: Revert "Add custom signup flow for donors" (T284740; T284800; T285281) (duration: 01m 06s)

Change 701361 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.9] Re-apply "Add custom signup flow for donors", step 1

https://gerrit.wikimedia.org/r/701361

Change 701363 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.9] Re-apply "Add custom signup flow for donors", step 2

https://gerrit.wikimedia.org/r/701363

Change 701364 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.9] Re-apply "Add custom signup flow for donors", step 3

https://gerrit.wikimedia.org/r/701364

Change 701361 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.9] Re-apply "Add custom signup flow for donors", step 1

https://gerrit.wikimedia.org/r/701361

Mentioned in SAL (#wikimedia-operations) [2021-06-24T11:34:41Z] <tgr@deploy1002> Synchronized php-1.37.0-wmf.9/extensions/GrowthExperiments: Backport: [[gerrit:701361|Re-apply "Add custom signup flow for donors", step 1 (T284799 T284740 T284800 T285281)]] (duration: 01m 06s)

Change 701363 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.9] Re-apply "Add custom signup flow for donors", step 2

https://gerrit.wikimedia.org/r/701363

Change 701364 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.9] Re-apply "Add custom signup flow for donors", step 3

https://gerrit.wikimedia.org/r/701364

Mentioned in SAL (#wikimedia-operations) [2021-06-24T12:08:28Z] <tgr@deploy1002> Synchronized php-1.37.0-wmf.9/extensions/GrowthExperiments: Backport: [[gerrit:701363|Re-apply "Add custom signup flow for donors", step 2 (T284799 T284740 T284800 T285281)]] (duration: 01m 06s)

Mentioned in SAL (#wikimedia-operations) [2021-06-24T12:18:43Z] <tgr@deploy1002> Started scap: Backport: [[gerrit:701364|Re-apply "Add custom signup flow for donors", step 3 (T284799 T284740 T284800 T285281)]]

Mentioned in SAL (#wikimedia-operations) [2021-06-24T12:44:50Z] <tgr@deploy1002> Finished scap: Backport: [[gerrit:701364|Re-apply "Add custom signup flow for donors", step 3 (T284799 T284740 T284800 T285281)]] (duration: 26m 07s)

@kostajh @Tgr @nettrom_WMF -- I just updated the description with the two URLs that Fundraising is going to use.

Change 701873 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[operations/mediawiki-config@master] GrowthExperiments: Set up campaign pattern

https://gerrit.wikimedia.org/r/701873

Change 701873 merged by jenkins-bot:

[operations/mediawiki-config@master] GrowthExperiments: Update campaign pattern

https://gerrit.wikimedia.org/r/701873

Mentioned in SAL (#wikimedia-operations) [2021-06-28T11:23:55Z] <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 9495d18521c3639cbd783a6341fe5348e31b0103: GrowthExperiments: Update campaign pattern (T284800) (duration: 00m 56s)

Change 701719 had a related patch set uploaded (by Kosta Harlan; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.11] Make it possible to force opt-in/opt-out to Growth features during account creation

https://gerrit.wikimedia.org/r/701719

Change 700671 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Make it possible to force opt-in/opt-out to Growth features during account creation

https://gerrit.wikimedia.org/r/700671

Change 701719 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.11] Make it possible to force opt-in/opt-out to Growth features during account creation

https://gerrit.wikimedia.org/r/701719

Mentioned in SAL (#wikimedia-operations) [2021-06-28T18:31:39Z] <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.11/extensions/GrowthExperiments/includes/HomepageHooks.php: ecf1d6c47c6cc30d84161e023373c6a2c7287be8: Make it possible to force opt-in/opt-out to Growth features during account creation (T284119; T284800; 1/3) (duration: 00m 58s)

Mentioned in SAL (#wikimedia-operations) [2021-06-28T18:32:59Z] <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.11/extensions/GrowthExperiments/includes/HelpPanelHooks.php: ecf1d6c47c6cc30d84161e023373c6a2c7287be8: Make it possible to force opt-in/opt-out to Growth features during account creation (T284119; T284800; 2/3) (duration: 00m 55s)

Mentioned in SAL (#wikimedia-operations) [2021-06-28T18:34:09Z] <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.11/extensions/GrowthExperiments/includes/WelcomeSurveyHooks.php: ecf1d6c47c6cc30d84161e023373c6a2c7287be8: Make it possible to force opt-in/opt-out to Growth features during account creation (T284119; T284800; 3/3) (duration: 00m 55s)

@Etonkovidova these are ready for testing (as a logged-out user):

After creating an account through those URLs you should have:

  • Homepage enabled
  • Help panel enabled
  • Suggested edits ready to set up
  • No welcome survey
  • growth-campaign user preference containing the value of the campaign query parameter, e.g. growth-advancement-spanish-2021

One note: This correctly leads me to the homepage right after creating my account, but when I leave it, this shows:

image.png (568×1 px, 167 KB)

I find it disruptive, since similar tour was already shown during homepage initial load.

Change 702358 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@master] Donor campaign: Don't show discovery tour to donor campaign users

https://gerrit.wikimedia.org/r/702358

Change 702358 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Welcome tour: Mark as complete when notice is shown

https://gerrit.wikimedia.org/r/702358

Change 702401 had a related patch set uploaded (by Gergő Tisza; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.11] Welcome tour: Mark as complete when notice is shown

https://gerrit.wikimedia.org/r/702401

Change 702402 had a related patch set uploaded (by Gergő Tisza; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.12] Welcome tour: Mark as complete when notice is shown

https://gerrit.wikimedia.org/r/702402

Change 702401 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.11] Welcome tour: Mark as complete when notice is shown

https://gerrit.wikimedia.org/r/702401

Change 702402 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.12] Welcome tour: Mark as complete when notice is shown

https://gerrit.wikimedia.org/r/702402

Mentioned in SAL (#wikimedia-operations) [2021-07-01T12:20:40Z] <tgr@deploy1002> Synchronized php-1.37.0-wmf.11/extensions/GrowthExperiments/: Backport: [[gerrit:702401|Welcome tour: Mark as complete when notice is shown (T284800)]] [[gerrit:702403|SuggestedEdits: Return default JS data as 'noresults' (T285906)]] (duration: 01m 09s)

Mentioned in SAL (#wikimedia-operations) [2021-07-01T12:22:56Z] <tgr@deploy1002> Synchronized php-1.37.0-wmf.12/extensions/GrowthExperiments/: Backport: [[gerrit:702402|Welcome tour: Mark as complete when notice is shown (T284800)]] [[gerrit:702404|SuggestedEdits: Return default JS data as 'noresults' (T285906)]] (duration: 01m 08s)

@Etonkovidova these are ready for testing (as a logged-out user):

After creating an account through those URLs you should have:

  • Homepage enabled
  • Help panel enabled
  • Suggested edits ready to set up
  • No welcome survey
  • growth-campaign user preference containing the value of the campaign query parameter, e.g. growth-advancement-spanish-2021

Checked - so far, works as expected.

Checked in wmf.14 - no issues have been found.

MMiller_WMF renamed this task from Donors to newcomers: URL parameters to Account creation: URL parameters.Feb 5 2022, 2:35 AM