No major updates. Continuing work on survey draft. Length will be limited to 15-17 questions (including age screener). Length variation depends solely on whether a modified series of urbanity questions are adopted (vs. the current single question format).

TBD:

• Best format for language questions (current variations in both CI and 2019 readers survey are unwieldy from UX perspective)
• Urbanity question (current CI and 2019 readers survey has "coverage hole" for middle-sized cities / suburban areas of said cities. See also this Pew note (although this research only applies to the United States)

Update:

Continuing to iterate on survey draft. Current working version here

Update:

Draft survey script has been completed and submitted for Legal review.

Update:

• Draft privacy statement has been received from Legal
• Continuing tweaks to draft survey script
• Preparing translation "drafts": documents that ease translation (see here for past examples)

Update:

• Workflow for translation has been determined and key decision points identified and described (see here: https://phabricator.wikimedia.org/T344947)
• Translation templates are being created for languages using Movement Comms translation

Update:

• English survey has been programmed in Limesurvey (testing underway)
• Translation templates created for Movement Comms translation (being reviewed)
• Reached out to Qualtrics rep for translation quote for Farsi, Simplified Chinese, and Dutch translations

Update: lots more translation prep happened this week!

• Translation templates completed (need to be reviewed again) for Movement Comms
• Translation templates for Qualtrics translation created
• Quatlrics survey for translation programmed
• English pilot quicksurvey deployment requested

Update:

• English pilot survey deployed with zero coverage: https://en.wikipedia.org/wiki/Wikimedia_Foundation?quicksurvey=reader-demographics-2
  • Will go "live" on September 25 with 1:300 coverage
  • See T345951 for details
• Translations updates (see T344947 for details):
  • requests submitted through Movement Comms process and are underway
  • Translations quote received from Qualtrics ($1500) and Coupa request submitted

Update:

• English pilot survey QS configuration updated to create a unique ID connecting QS logging with a prefilled Limesurvey variable (QS question in our Limesurvey programmed survey). Thank you to @Isaac for your invaluable assistance on this!
• Thanks to the whole Research team for serving as our survey testers!
  • Will continue to iterate on how best to solve for the bot problem while minimizing weird UX problems with Limesurvey text entry
• enwiki pilot survey ready to go live on September 25:
  • See T345951 for details
• Translations updates (see T344947 for details):
  • Most movement comms translations have been completed
  • Coupa request is being processed for Qualtrics translation.
  • Qualtrics contract has been signed
  • Qualtrics invoice has been forwarded to Merve Mursaloglu at Finance

Update:

• pilot survey fielded with 1/300 coverage on enwiki beginning on Monday September 25
• as of ~5:30 UTC on September 29, 1001 completes collected
• pilot survey to be undeployed on October 2
• Qualtrics translations (Farsi, Dutch, Simplified Chinese) to be completed and received by October 2 at latest

Update:

• pilot survey undeployed
• Qualtrics translations completed
• analysis is underway!

Update:

• Data cleaning and processing continues.
• Note for future reference that coding skipped questions is very onerous and complicated (since limesurvey does not currently record skips explicitly, may need to consider alternative survey programming practice—e.g., requiring response with prominent opt-out option)
  • almost all questions (apart from pseudo-CAPTCHA and 18+ question) currently offer opt-out option, but it can be made more prominent

Update:

• Translations are still missing for Hungarian and Vietnamese and incomplete for a few others; see translation status update in T344947 for more details
• Data cleaning and processing is complete for the enwiki pilot survey.
  • RE skip-coding: solution has been found using question-timing metadata. This should work in the future for all Limesurvey-hosted surveys
• Was able to calculate peak Limesurvey "usage" at 316 simultaneously open surveys. Tanja has contacted Limesurvey to see if they can share their data on this (would be useful to compare to see if our calculations are accurate).
  • Posted density plot shows that almost all surveys are open at a time when few other surveys are open.
  • Similarly, simple scatterplot of number of simultaneously open surveys against total time survey is open (each point is an individual survey instance) shows that most surveys are closed quickly and "high demand" appears disproportionately due to a relatively small number of surveys that are left open for long periods. This suggests we might want to have Limesurvey time out after 20-30 minutes (1200-1800 seconds).

Simple descriptives for extrapolating full sample survey demand:

{F38970374}
{F38973789}

Update:

• Full deployment is now anticipated to begin sometime after November 6.
• Survey script translations complete (see T344947 for details).
• Data pulled for updated coverage rate estimates here: https://docs.google.com/spreadsheets/d/1Z03-wFjIO29_6q9ycS4doZZoiVDylOamN54vdGFU3ng/edit#gid=1651585855
• Next steps are to program each language version into Limesurvey and to request zero-coverage Quicksurvey deployments

Update:

• Survey translations in all 24 versions programmed into limesurvey (see T350439)
• Projects prepared for Quicksurvey deployment T344393
• Projects planned for full deployment this week following testing!

Update:

• enwiki global readers survey launched this week at 2% coverage and is expected to complete data collection by November 23
• As of 4:31 PM on November 17, we have collected 4375 complete responses (16,118 surveys opened)
• the other 23 language versions will launch next week

Update:

• enwiki global readers survey completed data collection on November 22!
• 9476 completes, 8898 18+ completes.

Update:

• As detailed in T344393, global readers survey (non-english) launched on November 28
• Based on initial data collection November 28-29, surveys were relaunched November 29 at higher coverage for hiwiki, kowiki, elwiki, and idwiki
  • Note that we are continuing to face some challenges with hiwiki that may require further investigation: specifically, idiosyncratically low response rates among those clicking through from Quicksurvey

Update:

• Global readers survey data collection continues (currently planned to continue through December 15)

Update:

• Global readers survey collection closed on December 18. (T344393)
• Data cleaning and coding is underway and is expected to be completed in the first week of 2024 (T353843)
• Preliminary analysis of webrequest data by @Isaac on enwiki survey has begun and will continue (including all surveyed projects) in 2024.
• Project meta page has been updated with survey fielding details: https://meta.wikimedia.org/wiki/Research:Knowledge_Gaps_Index/Measurement/Readers_Survey_2023#Results

Update:

• survey data cleaning complete (T353843 for details)
• working on visualization code