Page MenuHomePhabricator

[EPIC] Diversity quicksurveys
Closed, DeclinedPublic

Description

Background

Research will be providing a series of baseline statistics on contributor diversity in one or more Wikimedia projects as a service to Audiences (TEC-9, Output 3.3).

They'd like to identify whether there are qualitative differences between participants of different gender and, if possible if this has any effects on their levels of participation. In particular, they'd like to answer the following questions:

  • How does participation vary by project or by country?
  • How does it vary by editing experience?
  • How does it vary in terms of time since first edit/registration?

In order to answer those questions, Research intends to target one or more surveys to users based in their location and by their edit count, which will require QuickSurveys' targetting to be updated.

Impact

  1. This work impacts T201707: Output 3.3: Baseline statistics on contributor diversity / TEC9
  2. This work impacts Output 1.3 of Communications' Every Human FY2018-19 program

Request Timeline

(As understood by @phuedx)

  1. TEC9, published 28th March, 2018, highlighted a need for "deep collaboration with teams in Audiences" (see https://www.mediawiki.org/wiki/Wikimedia_Technology/Annual_Plans/FY2019/TEC9:_Address_Knowledge_Gaps#Dependences).
  2. @JKatzWMF pinged @ovasileva and @phuedx about this work on 1st June, 2018
  3. @leila pinged @JKatzWMF (and mentioned @Jdlrobson) to discuss scheduling this work for Q2 FY2018-19 on September, 13th 2018 (T201707#4581861).
  4. The conversation around Output 1.3 of Communications' Every Human FY2018-19 program was reinvigorated (?) by Communications in December, 2018 and then again in February, 2019.

Event Timeline

ovasileva triaged this task as Medium priority.Feb 19 2019, 11:40 AM
ovasileva created this task.

Some quick documentation regarding the thinking behind the gender question:

There are two options for surveying with QuickSurveys: internal (single-question survey purely on wiki) and external (provider respondent with link to site like Google Forms with survey). For this gender survey, we are avoiding the external option for two reasons:

  • It has increased privacy concerns due to the survey being hosted off-wiki (often a necessary trade-off, but here we have a single question and a sensitive one for certain populations so we wish to avoid this)
  • It creates an additional burden to answering the survey. The respondent has to agree to take a survey and then load the survey in a new tab before they can respond. We have concerns that these additional clicks could substantially (and perhaps differentially) limit participation given the extra effort to respond. This would reduce our sample size as well as potentially introduce bias.

This leaves us with the internal survey option, which best preserves privacy and has a much lower burden to answering. Unfortunately the internal survey does not currently support open-text responses. Based on feedback and current best practices (described in T222744#5208134 regarding related reader surveys), it was decided that we cannot proceed with this survey without giving survey respondents the ability to self-identify their gender outside of our provided options. Thus, the functionality is currently being modified to allow for open-text responses per T225042.

Once this functionality is complete, we have a privacy policy prepared (https://foundation.wikimedia.org/wiki/2019_Editor_Gender_Survey_Privacy_Statement) and can begin the process of communication, deployment, and expansion to additional language communities.

Thanks for the great write up, @Isaac, and to Research for such deep consideration of how best to include everyone.