Page MenuHomePhabricator

Donors to newcomers: URL parameters
Open, HighPublic

Description

Donors will click on the call-to-action in their email and be brought to the wiki to create an account, hopefully to go on to edit. We want to be able to analyze the behavior of this cohort in all the ways that we usually analyze Growth newcomers through EventLogging and edit tags -- but we want to be able to separate out this cohort to see them distinctly.

We'll do this through identifying a URL query parameter in the call-to-action link. We may want to keep track of multiple query parameters if there are different subgroups in the cohort (e.g. new donors / repeat donors) or if we try different versions of the email. For example, the call to action link might be to https://es.wikipedia.org/wiki/Special:CreateAccount?campaign=growth-mexico-2021-a for variant A of an email or https://es.wikipedia.org/wiki/Special:CreateAccount?campaign=growth-mexico-2021-b for variant B, for an email campaign sent to donors in Mexico.

Engineering tasks

  • Hook into account creation process (like with the geForceVariant parameter) to look for the campaign parameter and then a known string pattern for the campaigns we want to track (e.g. growth-advancement-{blah-blah}
  • Create a hidden preference to store campaign URL query parameter for the user, for analytics purposes
  • Opt-in the user to Growth features (see also T284119: Growth: options for disabling/enabling Growth features for groups of users)

Event Timeline

MMiller_WMF moved this task from Inbox to Needs Discussion/Analysis on the Growth-Team board.

@kostajh -- could you specify in the task description what we'll need to know on the Growth team side in order to do this? So that Fundraising knows what is needed on their end. Thank you!

@nettrom_WMF -- I don't think you've heard much about these plans yet, but we're trying to move quickly on an idea with Fundraising. In short, we'll be emailing donors to ask them to create accounts. Then they'll create accounts, get picked up by Growth features, and hopefully go on to make their first edits. We need to be able to identify which accounts came through this workflow so that we can do our usual analytics, but restricted to just those accounts.

@kostajh proposes that we identify them using query parameters in the link they'll click in the email, followed by storing their status as a hidden preference. We would also use this query parameter to keep these participants out of our control groups, so that they all get the features.

As you think about this, what comes to mind? Do you foresee being able to easily work with these users via the hidden preference? Any concerns about keeping them out of control groups?

@MMiller_WMF : I set up a time for us to meet early next week to chat about this project so I don't miss anything important.

With regards to the idea to use the campaign query parameter at account creation and a hidden preference, that should work well to enable us to identify these accounts as it would mean that we can identify them when querying the Data Lake as well as the replicated databases.

I don't have immediate concerns about not having these accounts be randomly assigned to groups or not. The key parts to that are that 1) we document it so that we can exclude these from analysis where random assignment is expected, and 2) we'll be focusing on measurements related to conversion/participation to get a sense of success, rather than comparisons against a baseline (which a control group would've been).

Will being able to ensure that they're not in a control group mean that we're tying T284119 to this?

Lastly, if these users will see the Welcome Survey after account creation, the first question asks why they created an account today. None of the options appear to really fit this case, unless we're expecting them to have a second motivation?

I'll look a little more closely at this next time I have some time to spare for it, those where the things I could identify so far.

kostajh updated the task description. (Show Details)
kostajh edited projects, added Growth-Team (Current Sprint); removed Growth-Team.
kostajh edited subscribers, added: mewoph, DMburugu, Tgr; removed: nettrom_WMF, MeganHernandez_WMF, spatton and 5 others.

Change 700670 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@master] Make it possible to opt-out of Growth features

https://gerrit.wikimedia.org/r/700670

Change 700671 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@master] Make it possible for users to opt-out of Growth features

https://gerrit.wikimedia.org/r/700671

@nettrom_WMF and I talked about this today. We agreed that the plan laid out here (query parameter saved as hidden preference) is going to work for our purposes. We also reminded ourselves that these users will not be getting the welcome survey; we're going to send them straight from account creation to their homepage. Therefore, we don't want to record anything for their welcome survey data -- we should leave that blank.

Change 701097 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@master] Donor campaign: use hidden preference to store campaign

https://gerrit.wikimedia.org/r/701097

Change 701106 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/GrowthExperiments@master] Add custom signup flow for donors

https://gerrit.wikimedia.org/r/701106

Change 701097 abandoned by Gergő Tisza:

[mediawiki/extensions/GrowthExperiments@master] Donor campaign: use hidden preference to store campaign

Reason:

Squashed into Icfff8893f08d5ad1c161472210ad2707846074a8 for easier backporting.

https://gerrit.wikimedia.org/r/701097

Change 700670 abandoned by Urbanecm:

[mediawiki/extensions/GrowthExperiments@master] Make it possible to opt-out of Growth features

Reason:

Per discussion, let's go with I08ab0e4f9d instead.

https://gerrit.wikimedia.org/r/700670

Change 701106 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Add custom signup flow for donors

https://gerrit.wikimedia.org/r/701106

Change 700741 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.9] Add custom signup flow for donors

https://gerrit.wikimedia.org/r/700741

Change 700742 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.11] Add custom signup flow for donors

https://gerrit.wikimedia.org/r/700742

Change 700741 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.9] Add custom signup flow for donors

https://gerrit.wikimedia.org/r/700741

Change 700742 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.11] Add custom signup flow for donors

https://gerrit.wikimedia.org/r/700742

Mentioned in SAL (#wikimedia-operations) [2021-06-23T18:53:33Z] <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.11/extensions/GrowthExperiments/: 76e5fc91083736d14049a05ed227cdea015c113e: Add custom signup flow for donors (T284740; T284800; T285281) (duration: 01m 07s)

Mentioned in SAL (#wikimedia-operations) [2021-06-23T18:55:42Z] <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.9/extensions/GrowthExperiments/: REVERT: 76e5fc91083736d14049a05ed227cdea015c113e: Add custom signup flow for donors (T284740; T284800; T285281) (duration: 00m 38s)

Change 700744 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.9] Revert "Add custom signup flow for donors"

https://gerrit.wikimedia.org/r/700744

Change 700744 merged by Urbanecm:

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.9] Revert "Add custom signup flow for donors"

https://gerrit.wikimedia.org/r/700744

Mentioned in SAL (#wikimedia-operations) [2021-06-23T19:07:32Z] <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.9/extensions/GrowthExperiments/: 2338e53: Revert "Add custom signup flow for donors" (T284740; T284800; T285281) (duration: 01m 06s)