Page MenuHomePhabricator

Should be possible to sample by country
Closed, ResolvedPublic5 Estimated Story Points

Description

Add sampling by country options to QuickSurvey.

Precursors

An example survey will be necessary for us to build and test this survey.

Why?

  • Sometimes we only need data from a specific country or region and sampling from all requests to get enough number of responses from the region is not an option (we will be asking a lot of users to provide data that we won't use.)

What's already there?
Client-side country information is available under window.Geo.country

Acceptance criteria

  • A survey can be configured to target an editor in a certain country (or countries)

Notes about Implementation

  • Targeting a survey to a particular country requires including that country's two-character code in the configuration. It must be capitalized -- e.g., if you want to include the United States, you must have countries include "US"; none of "us" or "Us" or "USA" would work.
  • Not including countries in the configuration means country is not considered when targeting the survey
  • Including no countries in the configuration (i.e. "countries" => []) means no country is targeted (and therefore no one sees the survey)
  • Including one country in the configuration (e.g. "countries" => ["US"]) means that only users whose IP geocodes them to that country will be included
  • Including multiple countries in the configuration (e.g. "countries" => ["US", "CA"]) means that users from all of those countries can be shown the survey

The applicable two-letter country codes can be found in several places:

Sign off steps

We want to avoid the problem where when this survey goes live there is a problem with the implementation. We will want to sign this off with research carefully to avoid this happening.

Developer notes

Window.Geo is defined in the CentralNotice extension:
CentralNotice/modules/ext.centralNotice.bannerController/bannerController.js

QA steps

$.cookie( 'GeoIP', 'US:CO:Denver:39.6762:-104.887:v4' )
  • confirm survey shows
  • Make yourself Spanish:
$.cookie( 'GeoIP', 'ES:Catalonia:Barcelona:39.6762:-104.887:v4' )
  • confirm survey doesn't show
  • make yourself French
$.cookie( 'GeoIP', 'FR:Lille:39.6762:-104.887:v4' )
  • confirm survey is showing again

QA Results

StatusDetails
✅ PassedT213847#5028896

Event Timeline

ovasileva triaged this task as Medium priority.Feb 6 2019, 3:20 PM
ovasileva moved this task from Tracking to Upcoming on the Web-Team-Backlog board.
ovasileva edited projects, added Web-Team-Backlog; removed Web-Team-Backlog (Tracking).

I've pulled in T213696 to T139317. Because of the context switch any of these tasks involve it's probably good to think about the 3 of them (and possibly do them) together.

ovasileva set the point value for this task to 5.Feb 19 2019, 5:38 PM

Re: T213847#4986729:

  1. Nice find, @Jdlrobson!
  2. Should we discard surveys that required CentralNotice to be loaded if it isn't?

Re: T213847#4986729:

  1. Nice find, @Jdlrobson!
  2. Should we discard surveys that required CentralNotice to be loaded if it isn't?

We already use window.Geo
The Geo object is derived from a Cookie response header in the

  • CentralNotice ext.centralNotice.geoIP module (loaded on all
  • page views when installed). If the cookie was refused, this
  • falls back to the string "Unknown".

So looks like we can use it in the code provided we account for when it's not available.

Jdlrobson added subscribers: ovasileva, leila.

@ovasileva Stephen had a good question about this. Is it anticipated that surveys will be targetted to one country or multiple countries? Who is the customer for this task. Is it @leila? We need to ask a few more questions along this line.

@Jdlrobson feel free to ping @Isaac or myself here or in IRC.

Jumping in, I think multiple countries would be a good thing to plan for. For instance, if we want to survey users from a number of Latin American countries, we'd probably want to send the same survey to everyone but not write separate config files for each Spanish-speaking country. And Spain represents ~20% of the page views on eswiki [1] so not sampling by country would lead to a bunch of extra surveys going out.

And same thought process can be applied to regions in Africa that predominantly read English Wikipedia or French Wikipedia etc. where we may want to target 10+ countries at a time in a given language edition with the same survey.

[1] https://stats.wikimedia.org/wikimedia/animations/wivivi/wivivi.html

Change 494544 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/extensions/QuickSurveys@master] Surveys can be targetted at specific countries

https://gerrit.wikimedia.org/r/494544

Change 494544 merged by jenkins-bot:
[mediawiki/extensions/QuickSurveys@master] Surveys can be targeted at specific countries

https://gerrit.wikimedia.org/r/494544

Edtadros subscribed.

Test Result

Status: ✅ PASS
OS: macOS Mojave
Browser: Chrome DevTools Device Emulator (iPad Pro)

Test Artifact(s)/Step(s):

Test on DESKTOP
A survey has been setup to only show for users from America and France
Visit https://reading-web-staging.wmflabs.orgT/wiki/Test_the_diff
confirm survey doesn't show

T213847.png (2×2 px, 733 KB)

In your developer console make yourself "American" by running:
$.cookie( 'GeoIP', 'US:CO:Denver:39.6762:-104.887:v4' )
confirm survey shows

T213847-2.png (2×2 px, 745 KB)

Make yourself Spanish:
$.cookie( 'GeoIP', 'ES:Catalonia:Barcelona:39.6762:-104.887:v4' )
confirm survey doesn't show

T213847-3.png (2×2 px, 732 KB)

make yourself French
$.cookie( 'GeoIP', 'FR:Lille:39.6762:-104.887:v4' )
confirm survey is showing again

T213847-4.png (2×2 px, 745 KB)

This looks good to me. A few notes:

  • Targeting a survey to a particular country requires including that country's two-character code in the configuration. It must be capitalized -- e.g., if you want to include the United States, you must have countries include "US"; none of "us" or "Us" or "USA" would work.
  • Not including countries in the configuration means country is not considered when targeting the survey
  • Including no countries in the configuration (i.e. "countries" => []) means no country is targeted (and therefore no one sees the survey)
  • Including one country in the configuration (e.g. "countries" => ["US"]) means that only users whose IP geocodes them to that country will be included
  • Including multiple countries in the configuration (e.g. "countries" => ["US", "CA"]) means that users from all of those countries can be shown the survey

The applicable two-letter country codes can be found in several places:

Isaac updated the task description. (Show Details)

Just wanted to follow up w/ @Jdlrobson @ovasileva @phuedx @pmiazga @Edtadros (and anyone else I missed) to say thank you thank you for working on this!! Amongst others, we are currently running reader surveys in English and French (T212444) that are scoped to countries from Africa and getting thousands of responses, which is something we never would have been able to do without also getting literally hundreds of thousands of responses from the US, India, France, etc. that would be far more than we need.