Page MenuHomePhabricator

Run baseline quicksurvey on test wikis
Closed, ResolvedPublic3 Estimated Story Points

Description

Background

We would like to run a quicksurvey to establish a baseline on user affinity towards the desktop website. We will be repeating the survey at the end fo the desktop improvements project to see if there's any improvements in user affinity.

Acceptance criteria

  • Survey will run for one week on the following wikis:
    • English Wikipedia
    • French Wikipedia
    • French Wiktionary
    • Persian Wikipedia
    • Hebrew Wikipedia
  • Sampling rates (assuming 1% response rate)
wikisampling rate
enwiki0.25%
frwiki1.67%
fawiki17..1%
hewiki33.33%
frwiktionary20%
  • qualtrics links

available in: https://docs.google.com/document/d/1dDsi92WaYcw1R4C3kSPVurXdIXP7q3VHGrfWvnsjZjI/edit#

  • survey must include privacy statement

available in: https://docs.google.com/document/d/1dDsi92WaYcw1R4C3kSPVurXdIXP7q3VHGrfWvnsjZjI/edit#

  • survey must be turned off when we have enough results

Event Timeline

currently blocked on me putting questions into qualtrics

Suggested additions to wmf-config/InitialiseSettings.php:

wmf-config/InitialiseSettings.php
'wgQuickSurveysConfig' => [
  'enwiki' => [
    'enabled' => true,
    'type' => 'external',
    'name' => 'affinity-survey',
    'question' => 'ext-quicksurveys-affinity-survey-question',
    'description' => 'ext-quicksurveys-affinity-survey-description',
    'link' => 'https://wikimedia.qualtrics.com/jfe/form/SV_8Cv5UtY6Lby8j9H',
    'privacyPolicy' => 'ext-quicksurveys-affinity-survey-privacy-policy',
    'platforms' => [
      'desktop' => [ 'stable' ],
    ],
    'coverage' => 0.025,
  ],
  'frwiki' => [
    'enabled' => true,
    'type' => 'external',
    'name' => 'affinity-survey',
    'question' => 'ext-quicksurveys-affinity-survey-question',
    'description' => 'ext-quicksurveys-affinity-survey-description',
    'privacyPolicy' => 'ext-quicksurveys-affinity-survey-privacy-policy',
    'platforms' => [
      'desktop' => [ 'stable' ],
    ],
    'link' => '/* TODO */',
    'coverage' => 0.0167,
  ],
  'eswiki' => [
    'enabled' => true,
    'type' => 'external',
    'name' => 'affinity-survey',
    'question' => 'ext-quicksurveys-affinity-survey-question',
    'description' => 'ext-quicksurveys-affinity-survey-description',
    'privacyPolicy' => 'ext-quicksurveys-affinity-survey-privacy-policy',
    'platforms' => [
      'desktop' => [ 'stable' ],
    ],
    'link' => '/* TODO */',
    'coverage' => 0.0107,
  ],
  'fawiki' => [
    'enabled' => true,
    'type' => 'external',
    'name' => 'affinity-survey',
    'question' => 'ext-quicksurveys-affinity-survey-question',
    'description' => 'ext-quicksurveys-affinity-survey-description',
    'privacyPolicy' => 'ext-quicksurveys-affinity-survey-privacy-policy',
    'platforms' => [
      'desktop' => [ 'stable' ],
    ],
    'link' => '/* TODO */',
    'coverage' => 0.171,
  ],
  'hewiki' => [
    'enabled' => true,
    'type' => 'external',
    'name' => 'affinity-survey',
    'question' => 'ext-quicksurveys-affinity-survey-question',
    'description' => 'ext-quicksurveys-affinity-survey-description',
    'privacyPolicy' => 'ext-quicksurveys-affinity-survey-privacy-policy',
    'platforms' => [
      'desktop' => [ 'stable' ],
    ],
    'link' => '/* TODO */',
    'coverage' => 0.33,
  ],
  'frwiktionary' => [
    'enabled' => true,
    'type' => 'external',
    'name' => 'affinity-survey',
    'question' => 'ext-quicksurveys-affinity-survey-question',
    'description' => 'ext-quicksurveys-affinity-survey-description',
    'privacyPolicy' => 'ext-quicksurveys-affinity-survey-privacy-policy',
    'platforms' => [
      'desktop' => [ 'stable' ],
    ],
    'link' => '/* TODO */',
    'coverage' => 0.2,
  ],
],

For each of the target wikis we must define:

  • The ext-quicksurveys-affinity-survey-question message, which invites the user to take the survey
  • The 'ext-quicksurveys-affinity-survey-description message, which gives the user additional context
  • [Optional, possibly] The ext-quicksurveys-affinity-survey-privacy-policy message, which describes the survey's privacy policy to the user _briefly_ and/or provides a link to it
  • survey must include privacy statement. If available, privacy statement need to be in the language of the wiki. If not available, using the privacy statement in english is acceptable.

The privacy statement is a message. AIUI this message must be defined on the target wiki since it isn't defined in a message file in any of our codebases. Consequently, the message will not fallback to English.

If this behaviour is desirable, then we can define the message in the QuickSurveys codebase. However, this should be considered (minor) technical debt.

phuedx removed phuedx as the assignee of this task.Mar 9 2020, 6:35 PM
phuedx added a subscriber: phuedx.
ovasileva set the point value for this task to 3.Mar 10 2020, 5:27 PM

I've put this on the deploy calendar for Wed 11am.

I've put this on the deploy calendar for Wed 11am.

Perfect, thank you!

Mentioned in SAL (#wikimedia-operations) [2020-07-09T11:09:10Z] <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: e6f442c6900524482806aeb1b5162e65bf7c97ac: Enable Quicksurveys for Desktop Improvements Project (T246977) (duration: 01m 06s)

Change 610797 had a related patch set uploaded (by Jdrewniak; owner: Jdrewniak):
[operations/mediawiki-config@master] Updating config for Readers Web affinity quicksurvey

https://gerrit.wikimedia.org/r/610797

Change 610797 merged by jenkins-bot:
[operations/mediawiki-config@master] Updating config for Readers Web affinity quicksurvey

https://gerrit.wikimedia.org/r/610797

Mentioned in SAL (#wikimedia-operations) [2020-07-09T18:10:10Z] <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 9f2557f848e99facaa62ca6b3a948cc3e32c32a3: Updating config for Readers Web affinity quicksurvey (T246977) (duration: 01m 06s)

Change 612870 had a related patch set uploaded (by Jdrewniak; owner: Jdrewniak):
[operations/mediawiki-config@master] Disable affinity quicksurveys for the following wikis

https://gerrit.wikimedia.org/r/612870

Change 612870 merged by jenkins-bot:
[operations/mediawiki-config@master] Disable affinity quicksurveys for the following wikis

https://gerrit.wikimedia.org/r/612870

Mentioned in SAL (#wikimedia-operations) [2020-07-16T18:08:11Z] <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:612870]] T246977 Disable affinity quicksurveys for the following wikis (duration: 00m 57s)

Surveys are disabled - resolving this!