Page MenuHomePhabricator

enable opt-in checkbox on the Welcome Survey allowing new account holders to consent to being contacted for design research
Closed, ResolvedPublic

Description

The WMF Design Strategy team would like to enable an opt-in checkbox on the Welcome Survey to allow newly-registered, email-providing users to consent to being contacted for design research recruitment. Currently, this desire is in support of Growth’s Non-Editing Participation work.

A similar but unrelated effort is described in T304422.

Ideally, we’d like this checkbox to be toggleable for this specific project, and hopefully for future individual projects as well if there is agreement about this plan. If possible, we would like to enable this checkbox for the following languages for the NEP research project:

  • English
  • Arabic
  • Spanish

Emails sent to CSV

  • T304422 requested the following maintenance script features:
  • support querying by dates with --from and --to fields
  • output-mode, default is 'text' to print to stdout (one entry per line), otherwise --output-mode=csv would create a CSV
  • allow outputting data on whether the user has a confirmed email or not
  • allow outputting null value for email in case the user has checked the opt-in box but doesn't have a supplied email (for QA purposes, really)

WMF Legal approval

  • WMF Legal has expressed their provisional approval for this plan, provided the following guidelines are met:
  • Each time the checkbox is enabled, it is supported by a privacy statement for that specific project.
  • Emails are stored in a secure place
  • Emails and other data can be deleted in a timely manner
  • Emails and other data are shared in as limited way as possible internally (i.e., not being shared beyond the project members immediately involved in recruitment)
Design

Figma designs for Welcome survey checkbox

MobileDesktop
Mobile-WelcomeSurvey.png (1×362 px, 92 KB)
Desktop-WelcomeSurvey.png (925×1 px, 181 KB)
Acceptance Criteria:

Given I'm a new account holder who has provided an email address,
When I am filling out the welcome survey

  • And we have enabled the opt-in checkbox for my home wiki,

Then there is a an optional opt-in checkbox on the Welcome Survey allowing me to consent to being contacted for design research

NOTE: we should show a link to the Welcome Survey privacy statement from the welcome survey even if we aren't including the opt-in checkbox.

Given I'm viewing the Welcome survey (and I have provided an email OR NOT provided an email)
When I am filling out the welcome survey

  • And we have NOT enabled the opt-in checkbox for my home wiki,

Then rather than an opt-in checkbox I see the following text:

The Wikimedia Foundation will use your answers to help you start editing Wikipedia. Learn more in our privacy statement.


Given I'm WMF staff,
When I have securely logged in and have access to private data,
Then I have access to a list of newcomers email addresses who have opted in:

  • support querying by dates with --from and --to fields
  • output-mode, default is 'text' to print to stdout (one entry per line), otherwise --output-mode=csv would create a CSV
  • allow outputting data on whether the user has a confirmed email or not
  • allow outputting null value for email in case the user has checked the opt-in box but doesn't have a supplied email (for QA purposes, really)
  • Emails are stored in a secure place
  • Emails and other data are deleted in a timely manner (TBD: 90 days?)

We will need to be able to limit this opt-in checkbox to specific wikis; Design research will decide in advance which wikis we want to target.

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 946512 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@master] [WIP] WelcomeSurvey: Add user-research question to the bank

https://gerrit.wikimedia.org/r/946512

Hi @MRaishWMF @KStoller-WMF, thanks for the additional information. I only need the link to the privacy statement, so it can be linked from the interface. Once that is provided and added to the patch, we can go to code review I believe.

I chatted with @MRaishWMF about the project's needs, and here is a summary of the discussion:

  • The project is of a short duration and there is no need for a continuous data access. On-demand export using the maintenance script we created in T304422 should work.
  • There might be more projects that would use the checkbox. Each one will have a separate privacy statement.
  • At least a week before each experiment starts, Mike/Design Research will share the privacy statement link. Then, a new question can be added to the question bank, verified it works and eventually enabled in production.

While working on this, I noticed a possible issue in Welcome survey's existing privacy links. Filled as T343682: Welcome survey: The link under "we value your privacy" is outdated.

Change 946514 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@master] Generalize ExportWelcomeSurveyMailingListData maint script

https://gerrit.wikimedia.org/r/946514

I've received the privacy statement link from @MRaishWMF: https://foundation.wikimedia.org/wiki/Legal:Non_Editing_Participation:_Looking_at_Examples_Survey_Privacy_Statement.

This is not ready to deploy yet, but it is ready for preparing the WelcomeSurvey for the new question. Next step is reviewing the patches I already uploaded. Ideally, we can complete T344262 in the meantime and then proceed to actually change the configuration.

Change 946514 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Generalize ExportWelcomeSurveyMailingListData maint script

https://gerrit.wikimedia.org/r/946514

Change 949849 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[operations/mediawiki-config@master] [beta] Growth: Enable user research opt-in checkbox on few wikis

https://gerrit.wikimedia.org/r/949849

This now depends on Code Review. Once merged, the next steps is to test on beta and then wait for the "go-to-production" signal from @MRaishWMF.

Change 946512 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] WelcomeSurvey: Add user-research question to the bank

https://gerrit.wikimedia.org/r/946512

Change 949849 merged by jenkins-bot:

[operations/mediawiki-config@master] [beta] Growth: Enable user research opt-in checkbox on few wikis

https://gerrit.wikimedia.org/r/949849

@MRaish_WMF This is now deployed at the beta cluster (arwiki, enwiki and eswiki, as requested). Please have a look and let me know if there need to be any changes (apart from translations, which can be provided via Translatewiki.net).

Change 951151 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[operations/mediawiki-config@master] Growth: Remove wgWelcomeSurveyEnableWithHomepage

https://gerrit.wikimedia.org/r/951151

Change 951151 merged by jenkins-bot:

[operations/mediawiki-config@master] Growth: Remove wgWelcomeSurveyEnableWithHomepage

https://gerrit.wikimedia.org/r/951151

Mentioned in SAL (#wikimedia-operations) [2023-08-21T20:21:27Z] <urbanecm@deploy1002> Started scap: Backport for [[gerrit:951151|Growth: Remove wgWelcomeSurveyEnableWithHomepage (T342353 T344619)]], [[gerrit:950812|revalidateLinkRecommendations: Load scoreLessThan correctly (T316079)]], [[gerrit:950813|LinkRecommendationUpdater: Load link-recommendation even if disabled (T344343)]]

Mentioned in SAL (#wikimedia-operations) [2023-08-21T20:23:01Z] <urbanecm@deploy1002> urbanecm: Backport for [[gerrit:951151|Growth: Remove wgWelcomeSurveyEnableWithHomepage (T342353 T344619)]], [[gerrit:950812|revalidateLinkRecommendations: Load scoreLessThan correctly (T316079)]], [[gerrit:950813|LinkRecommendationUpdater: Load link-recommendation even if disabled (T344343)]] synced to the testservers mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet, mwde

Mentioned in SAL (#wikimedia-operations) [2023-08-21T20:32:29Z] <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:951151|Growth: Remove wgWelcomeSurveyEnableWithHomepage (T342353 T344619)]], [[gerrit:950812|revalidateLinkRecommendations: Load scoreLessThan correctly (T316079)]], [[gerrit:950813|LinkRecommendationUpdater: Load link-recommendation even if disabled (T344343)]] (duration: 11m 02s)

@MRaishWMF pointed out the survey didn't work as expected on enwiki beta. That should be fixed now. Waiting on review.

Reassigning for the reviews (and translations, which would need to exist for non-beta deployment). Feel free to assign back to me once the survey's reviewed, ready to go and translated!

@Urbanecm_WMF, I can add a new task if you prefer, but I figured it might be best to keep this all together. The request from Legal is that we provide a link to the Welcome Survey privacy statement from the welcome survey even if we aren't including the opt-in checkbox.

So I added this acceptance criteria:
Given I'm viewing the Welcome survey (and I have provided an email addresss OR NOT provided an email address)
When I am filling out the welcome survey

  • And we have NOT enabled the opt-in checkbox for my home wiki,

Then rather than an opt-in checkbox (and associated text) I see the following text:

Please see our privacy statement for this survey.

Sorry, for all of the changes and confusion around this task!

Hi @KStoller-WMF, thanks for the info! I think we should centralize the privacy statement changes in the other task, T343682. I've included screenshot examples at T343682#9113149.

Testing the survey is currently blocked by T344834.

Testing the survey is currently blocked by T344834.

This is now resolved.

Change 952347 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@master] Welcome survey: Add generic user-research question

https://gerrit.wikimedia.org/r/952347

WMF Legal mentioned the research-specific privacy statement (in this run, https://foundation.wikimedia.org/wiki/Legal:Non_Editing_Participation:_Looking_at_Examples_Survey_Privacy_Statement) doesn't need to be included in Growth's Welcome survey directly, and that it is sufficient for that link to be included only when we reach out to the users.

After a discussion with Mike, I decided to go ahead with the current survey in the first round. We can use the generic question in a future round.

Change 952351 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[operations/mediawiki-config@master] Growth: Welcome survey user research: Use a generic question

https://gerrit.wikimedia.org/r/952351

Change 952347 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Welcome survey: Add generic user-research question

https://gerrit.wikimedia.org/r/952347

Change 952351 merged by jenkins-bot:

[operations/mediawiki-config@master] Growth: Welcome survey user research: Use a generic question

https://gerrit.wikimedia.org/r/952351

Change 964013 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[operations/mediawiki-config@master] Growth: Enable Welcome survey user research for enwiki

https://gerrit.wikimedia.org/r/964013

Per @MRaishWMF's request, I've scheduled a deployment of this for Tuesday Oct 10. We'll enable the survey for English Wikipedia only, and revert on the next day. Using the acquired data, we'll test if everything works as expected and also to better gauge the number of responses we'll be getting.

Change 964013 merged by jenkins-bot:

[operations/mediawiki-config@master] Growth: Enable Welcome survey user research for enwiki

https://gerrit.wikimedia.org/r/964013

Mentioned in SAL (#wikimedia-operations) [2023-10-10T13:26:51Z] <urbanecm@deploy2002> Started scap: Backport for [[gerrit:964013|Growth: Enable Welcome survey user research for enwiki (T342353)]]

Mentioned in SAL (#wikimedia-operations) [2023-10-10T13:28:09Z] <urbanecm@deploy2002> urbanecm: Backport for [[gerrit:964013|Growth: Enable Welcome survey user research for enwiki (T342353)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2023-10-10T13:40:11Z] <urbanecm@deploy2002> Finished scap: Backport for [[gerrit:964013|Growth: Enable Welcome survey user research for enwiki (T342353)]] (duration: 13m 19s)

Change 965219 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[operations/mediawiki-config@master] Revert "Growth: Enable Welcome survey user research for enwiki"

https://gerrit.wikimedia.org/r/965219

Change 965219 merged by jenkins-bot:

[operations/mediawiki-config@master] Revert "Growth: Enable Welcome survey user research for enwiki"

https://gerrit.wikimedia.org/r/965219

Mentioned in SAL (#wikimedia-operations) [2023-10-12T20:41:50Z] <dr0ptp4kt@deploy2002> Started scap: Backport for [[gerrit:965219|Revert "Growth: Enable Welcome survey user research for enwiki" (T342353)]]

Mentioned in SAL (#wikimedia-operations) [2023-10-12T20:43:06Z] <dr0ptp4kt@deploy2002> dr0ptp4kt and urbanecm: Backport for [[gerrit:965219|Revert "Growth: Enable Welcome survey user research for enwiki" (T342353)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2023-10-12T20:50:22Z] <dr0ptp4kt@deploy2002> Finished scap: Backport for [[gerrit:965219|Revert "Growth: Enable Welcome survey user research for enwiki" (T342353)]] (duration: 08m 32s)

Functionally done, verified in enwiki, first export is ready. Pending full deployment once @MRaish_WMF is ready.

Based on a discussion with @MRaishWMF, I'm closing this task as Resolved, as we both (a) made it possible to enable the user research checkbox on certain wikis (b) did a short deployment on English Wikipedia. Mike is currently working with the exported email addresses.

We will deploy the same checkbox to some other wikis (probably enwiki/arwiki) later on; when the time comes, a new task will be created. This is to make task management easier, as the new task would be a simple request, with no preparation needs.

Thanks everyone!