Page MenuHomePhabricator

WE 3.6.8 Comparative Reader Research
Closed, ResolvedPublic

Description

This tracks work done under the WE 3.6.8 "Comparative Reader Research" Hypothesis. This work is also tracked on Asana.

Hypothesis text:
If we systematically compare the information seeking needs and behaviors of 1) new and infrequent English Wikipedia readers and non-readers with those of 2) current English Wikipedia readers by simultaneously surveying both populations, we will be able to identify key insights about the demographic characteristics, online information needs, and online platforms used by these audiences, identifying potential opportunities for growing our readership as well as potential threats to our existing readership.

Research Goals:
This research has the following goals:
Identify opportunities for growing readership:
Provide representative data on the size (relative to Wikipedia readership in the Prolific survey sample), demographic characteristics, and online information needs of potential growth audiences for English Wikipedia
Provide representative data on the other platforms used by growth audiences to find information online and the needs these platforms serve for them
Identify threats to existing readership:
Provide representative data on the demographic characteristics and online information needs of current English and Spanish Wikipedia readers
Provide representative data on the other platforms used by current English Wikipedia readers to find information online and the needs these platforms serve for them
Methods:
This research will survey simultaneous samples of English-language internet users (sourced from Prolific) and English and Spanish Wikipedia readers (randomly-sampled from English and Spanish Wikipedia using the Quicksurvey tool).

Surveys will be administered via Qualtrics.

Survey draft

Timeline:
We expect this work (surveys to be drafted, fielded, data collected, data analyzed, and results reported) to be concluded by the end of Q3 (March 31, 2026).

  • survey draft to be shared following the FA/WE3 offsite (circulated with core stakeholders by Feb 13).
  • survey to field week of February 23
  • survey data collection to complete by March 4
  • initial data analysis shared by March 20
  • full presentation and reporting of survey data analysis to be completed by end of Q3
    • presentation to larger readers staff group on April 16

Details

Other Assignee
MRaishWMF

Event Timeline

Update:

  • Research team for this project has been assembled and roles and responsibilities assigned.
  • Initial project timeline has been developed.
  • Survey drafting is underway

Weekly update:

  • Coordinating an unconference session at upcoming WE3/FA offsite for input to survey draft.
  • Survey drafting continues

Weekly update:

  • Survey draft to be shared early next week.
  • Decision to be made early next week on Qualtrics or Limesurvey administration
  • Tentative plan to field in English and in Spanish

Weekly update:

  • survey draft shared with stakeholders, input incorporated into draft.
  • qualtrics will be used to field survey.
  • will attempt to field ASAP once survey draft finalized and approved by stakeholders
  • survey privacy statement requested and received

Weekly update:

  • survey draft approved
  • survey revised based on feedback and on initial user testing in Userlytics
  • english survey draft finalized and sent out for translation
  • QS logistics prep underway
  • english survey draft programmed in Qualtrics

Weekly update:

  • Surveys fielded on-platform on English and Spanish Wikipedias
  • Responses are tracking for undeployment on Monday March 2.
  • Adopted staggered survey fielding schedule on Prolific based on screening / pilot results suggesting that:
    • Most respondents recruited through Prolific self-report visiting Wikipedia at least once in the past 30 days (this persists even when we introduce messaging normalizing not reading Wikipedia)
    • Extended pilot testing and additional screening means we have to increase budget for this project (add $ to our Prolific account)
  • Staggered survey fielding schedule on Prolific:
    • Nationally-representative surveyed in English for US and UK (launched 2/26, mostly complete as of 2/27)
    • Spanish survey launched 2/27
    • Oversample of non- / infrequent Wikipedia readers to field March 2 in English and Spanish (as needed). We will screen respondents based on their self-reported Wikipedia reading and only contact those who say they haven’t visited in the past 30 days.
      • Methods note: this means that Wikipedia reading frequency will not be “representative” of even Prolific panelists, but we want to ensure that we have a substantial number of non- / infrequent readers
      • If necessary, Spanish oversample may also include geographic oversamples to ensure reasonable sample sizes in top Spanish Wikipedia markets (Spain, Mexico, southern Cone, Colombia/Venezuela)
  • Preliminary data cleaning and analysis is underway to facilitate oversampling / speed up final analysis process

Weekly update:

  • Data collection has been completed on:
    1. English on-platform survey sample
    2. Spanish on-platform survey sample
    3. US/UK nationally-representative English language Prolific samples
    4. “Open” - no demographic or geographic requirements - English language Prolific sample
    5. “Open” - no demographic or geographic requirements - Spanish language Prolific sample
  • We recommend closing data collection
    • Successfully collected substantial survey samples of both on- and off-platform audiences
    • Oversamples of those self-identifying as non-readers (have not visited in past 30 days) in English and Spanish have yielded subsamples of non-readers that are sufficiently large to enable subgroup analysis:
      • English Prolific non-reader respondents (n≈350)
      • Spanish Prolific non-reader respondents (n≈250)
    • Geographic distribution of Spanish Prolific respondents is different from that of Spanish Wikipedia readership, but provides reasonable representation of major geographic blocs (ie, Spain, Mexico, Colombia/Venezuela, southern cone of South America). Note however that Prolific respondents from southern cone are very disproportionately from Chile.

Weekly update:

  • Data cleaning completed and survey response analysis and visualization process is underway
  • On-platform responses matched to Quicksurvey initiations. Weighting and webrequest data analysis to come next.

Weekly update:

  • Analysis continues
  • Draft deck sharing results expected by end of next week
YLiou_WMF added a subscriber: ovasileva.

Weekly update:

Update:

  • Results delivered to @ovasileva.
  • Shareout to all-readers staff group scheduled for April 16.
YLiou_WMF updated the task description. (Show Details)