Page MenuHomePhabricator

Should be possible to sample by country
Closed, ResolvedPublic5 Story Points

Description

Add sampling by country options to QuickSurvey.

Precursors

An example survey will be necessary for us to build and test this survey.

Why?

  • Sometimes we only need data from a specific country or region and sampling from all requests to get enough number of responses from the region is not an option (we will be asking a lot of users to provide data that we won't use.)

What's already there?
Client-side country information is available under window.Geo.country

Acceptance criteria

  • A survey can be configured to target an editor in a certain country (or countries)

Notes about Implementation

  • Targeting a survey to a particular country requires including that country's two-character code in the configuration. It must be capitalized -- e.g., if you want to include the United States, you must have countries include "US"; none of "us" or "Us" or "USA" would work.
  • Not including countries in the configuration means country is not considered when targeting the survey
  • Including no countries in the configuration (i.e. "countries" => []) means no country is targeted (and therefore no one sees the survey)
  • Including one country in the configuration (e.g. "countries" => ["US"]) means that only users whose IP geocodes them to that country will be included
  • Including multiple countries in the configuration (e.g. "countries" => ["US", "CA"]) means that users from all of those countries can be shown the survey

The applicable two-letter country codes can be found in several places:

Sign off steps

We want to avoid the problem where when this survey goes live there is a problem with the implementation. We will want to sign this off with research carefully to avoid this happening.

Developer notes

Window.Geo is defined in the CentralNotice extension:
CentralNotice/modules/ext.centralNotice.bannerController/bannerController.js

QA steps

$.cookie( 'GeoIP', 'US:CO:Denver:39.6762:-104.887:v4' )
  • confirm survey shows
  • Make yourself Spanish:
$.cookie( 'GeoIP', 'ES:Catalonia:Barcelona:39.6762:-104.887:v4' )
  • confirm survey doesn't show
  • make yourself French
$.cookie( 'GeoIP', 'FR:Lille:39.6762:-104.887:v4' )
  • confirm survey is showing again

QA Results

StatusDetails
✅ PassedT213847#5028896

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 15 2019, 5:58 PM
phuedx added a subscriber: phuedx.Jan 25 2019, 1:53 PM

I've added T203042: Output 2.2: Characterizing readership by demographics as a parent task as this task was split out of T213696: Sample by title in QuickSurvey (see T213696#4881990) and has it as its parent.

ovasileva triaged this task as Normal priority.

I've pulled in T213696 to T139317. Because of the context switch any of these tasks involve it's probably good to think about the 3 of them (and possibly do them) together.

ovasileva set the point value for this task to 5.Feb 19 2019, 5:38 PM

Re: T213847#4986729:

  1. Nice find, @Jdlrobson!
  2. Should we discard surveys that required CentralNotice to be loaded if it isn't?

Re: T213847#4986729:

  1. Nice find, @Jdlrobson!
  2. Should we discard surveys that required CentralNotice to be loaded if it isn't?

We already use window.Geo
The Geo object is derived from a Cookie response header in the

  • CentralNotice ext.centralNotice.geoIP module (loaded on all
  • page views when installed). If the cookie was refused, this
  • falls back to the string "Unknown".

So looks like we can use it in the code provided we account for when it's not available.

Jdlrobson added subscribers: ovasileva, leila.

@ovasileva Stephen had a good question about this. Is it anticipated that surveys will be targetted to one country or multiple countries? Who is the customer for this task. Is it @leila? We need to ask a few more questions along this line.

leila added a subscriber: Isaac.Feb 27 2019, 7:23 PM

@Jdlrobson feel free to ping @Isaac or myself here or in IRC.

Isaac added a comment.Mar 1 2019, 4:34 PM

Jumping in, I think multiple countries would be a good thing to plan for. For instance, if we want to survey users from a number of Latin American countries, we'd probably want to send the same survey to everyone but not write separate config files for each Spanish-speaking country. And Spain represents ~20% of the page views on eswiki [1] so not sampling by country would lead to a bunch of extra surveys going out.

And same thought process can be applied to regions in Africa that predominantly read English Wikipedia or French Wikipedia etc. where we may want to target 10+ countries at a time in a given language edition with the same survey.

[1] https://stats.wikimedia.org/wikimedia/animations/wivivi/wivivi.html

Change 494544 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/extensions/QuickSurveys@master] Surveys can be targetted at specific countries

https://gerrit.wikimedia.org/r/494544

Change 494544 merged by jenkins-bot:
[mediawiki/extensions/QuickSurveys@master] Surveys can be targeted at specific countries

https://gerrit.wikimedia.org/r/494544

Jdlrobson assigned this task to Edtadros.
Edtadros reassigned this task from Edtadros to ovasileva.
Edtadros added a subscriber: Edtadros.

Test Result

Status: ✅ PASS
OS: macOS Mojave
Browser: Chrome DevTools Device Emulator (iPad Pro)

Test Artifact(s)/Step(s):

Test on DESKTOP
A survey has been setup to only show for users from America and France
Visit https://reading-web-staging.wmflabs.orgT/wiki/Test_the_diff
confirm survey doesn't show

In your developer console make yourself "American" by running:
$.cookie( 'GeoIP', 'US:CO:Denver:39.6762:-104.887:v4' )
confirm survey shows

Make yourself Spanish:
$.cookie( 'GeoIP', 'ES:Catalonia:Barcelona:39.6762:-104.887:v4' )
confirm survey doesn't show

make yourself French
$.cookie( 'GeoIP', 'FR:Lille:39.6762:-104.887:v4' )
confirm survey is showing again

phuedx removed a subscriber: phuedx.Mar 18 2019, 9:21 AM
ovasileva reassigned this task from ovasileva to Isaac.Mar 20 2019, 9:38 AM
Isaac added a comment.Mar 21 2019, 4:19 PM

This looks good to me. A few notes:

  • Targeting a survey to a particular country requires including that country's two-character code in the configuration. It must be capitalized -- e.g., if you want to include the United States, you must have countries include "US"; none of "us" or "Us" or "USA" would work.
  • Not including countries in the configuration means country is not considered when targeting the survey
  • Including no countries in the configuration (i.e. "countries" => []) means no country is targeted (and therefore no one sees the survey)
  • Including one country in the configuration (e.g. "countries" => ["US"]) means that only users whose IP geocodes them to that country will be included
  • Including multiple countries in the configuration (e.g. "countries" => ["US", "CA"]) means that users from all of those countries can be shown the survey

The applicable two-letter country codes can be found in several places:

Isaac closed this task as Resolved.Mar 21 2019, 4:21 PM
Isaac updated the task description. (Show Details)