Project goal

In 2025, we want to experiment with changing the recruitment and response methodology for the biennial Community Insights survey (scheduled for the next full data collection in 2026).

This change would entail shifting from our current process of querying the eligible editor population and sending unique survey links via EmailUser with a two-step email opt-in process, to directly sampling on the projects using QuickSurvey to link to a LimeSurvey-based survey for those who agree to participate.

Some benefits of this approach are to reduce researcher and analyst burden from sample drawing and survey distribution & monitoring, allow us to shift to a FLOSS survey tool, and reduce respondent burden by making the opt-in a single-step on-wiki process. Potential benefits are to reduce nonresponse bias by simplifying the opt-in process while increasing the potential respondent size to make better project-based estimates.

The potential risks of this approach are that data may not be exactly comparable to past years as our methodology for sampling and parameters for eligibility would be different, and that QuickSurveys currently only works on Wikipedia projects. The aim of this test is to see whether we can collect comparable data, and as many Community Insights responses do not change drastically over time (indicating stable data quality), dramatic changes across responses compared to the previous survey would indicate the population being sampled is potentially too different from the population of previous surveys. As English Wikipedia has the largest editor population (and thus largest population in previous Community Insights surveys), we can start with this project to test viability for expanding to the rest of the projects.

Whether or not we find the data is comparable, we can still gain valuable insights from this test survey for the population frame surveyed. We can also take this as an opportunity to pilot new questions that are timely for current initiatives.

Tasks

Forthcoming.