Page MenuHomePhabricator

Lead interview portion of project inquiry
Closed, ResolvedPublic

Description

With support from the group, lead the interview portion of the study. A breakdown of this work would include:

Preliminary work (8 August - 27 September)

  • Contributing to sampling approach and distribution method (currently ongoing)
  • Develop recruitment plan (identify key characteristics, primary and fallback recruitment approaches)
  • Develop discussion guide for interviews
  • Obtain privacy statement for interviews
  • Localize relevant materials and arrange for interpretation where necessary

Data collection (27 September - 27 November)

  • Ongoing recruitment efforts
  • Schedule and coordinate interviews (with additional support as necessary)
  • Preliminary data organization

Analysis (1 Demecber - 20 December)

  • Synthesis of interview data
  • Coordinate with other project leads during analysis process
  • Creation of deliverables associated with the interview portion of the project

Reporting and socialization of results (6 - 20 December)

  • Finalize report/deck
  • Presentation/discussion with stakeholders

Details

Due Date
Dec 1 2024, 6:00 AM

Event Timeline

leila triaged this task as High priority.Aug 14 2024, 4:00 PM
cwylo set Due Date to Nov 11 2024, 6:00 AM.
Easikingarmager changed the task status from Open to In Progress.EditedAug 21 2024, 8:25 PM
Easikingarmager updated the task description. (Show Details)

I added our targeted phase dates to sync these activities with the timeline at: https://meta.wikimedia.org/wiki/Research:Wikipedia_Administrator_Recruitment,_Retention,_and_Attrition#Timeline. I also added the final reporting/socialization phase and dates to capture work at that time.

Weekly update:

  • Recruitment strategy and timelines for the survey and interviews
  • Began work on writing the interview discussion guide (internal for now)
  • Working with Bethany to create logistics plan for the interview and survey
cwylo moved this task from In Progress to Staged on the Research board.

Belated weekly update: continued development of the discussion guide. The guide will essentially be split into a "current admin/survey participant" version, and a "non-participant/ex-admin" version, since those are our two key groups for recruitment. The aim is to have interview materials ready in parallel with the survey materials, so that feedback, privacy review, and other logistical tasks can be handled simultaneously with the survey logistical requirements.

Update:

  • Discussion guide finalized (internal link)
  • Outreach emails, scheduler tool, and adapted privacy release form ready for translation (thanks to Bethany!)

Our recruitment plan remains the same - we will start by recruiting ex-admins. We will wait until the survey is live and we begin obtaining preliminary results before recruiting survey participants for our "current administrator" and "prospective administrator" section. Dates remain accurate, and we're on track to hit them.

Update:

  • Assisted with survey piloting efforts
  • Consultations with staff members who work closely with ru, fr, id Wikipedia communities, to better prepare for interview recruitment and discussion plans
  • Preparing recruitment for ex-administrator segment

Update:

  • Reached out to Movement Comms to assist with recruitment
  • Translations received for outreach messages, localization of outreach messages, scheduling tool, and privacy releases completed
  • Recruitment has started for the enwiki former administrator segment
  • Recruitment slated to begin for idwiki and frwiki former administrators
cwylo changed Due Date from Nov 11 2024, 6:00 AM to Nov 27 2024, 6:00 AM.

Recruitment efforts ongoing for the former admin segment.

  • We have currently completed one interview
  • Three more are scheduled for the upcoming week
  • We have had one no-show interview so far

Given the delays for survey deployment, and relatively low response rates so far, I believe it makes the most sense to begin interview recruitment for current administrators in advance of survey deployment where possible. Our data collection deadline has been pushed back to 27 Nov to reflect the current status of the project.

Update:

  • We have completed three more interviews this week, with another scheduled
  • Two more interviews are being scheduled
  • We are preparing to interview survey respondents who indicate interest in being contacted (T371135#10263674)

So far, we have hit our target of two former admins per wiki with the exception of Spanish and French Wikipedias. We intend to weight recruitment from the current admin segment more heavily towards them, to help remedy the gap.

In preparation for my upcoming OOO, we have also discussed logistical procedures. Eli and Bethany will split responsibilities for scheduling and interviewing participants, and I will be able to review notes and recordings asynchronously when I return.

Hi @dchen, since you weren't yet subscribed, I wanted to tag you here on the relevant phab task that you've started helping with. Thanks for your work getting up to speed this week and helping us!

Post-offsite update:

  • Talked to Daisy and Bethany about current interview status - we have four more interviews lined up
  • Our final "missing" segments are current French and Spanish Wikipedia admins. I will be leading the efforts to schedule interviews for this segment
  • Due to time constraints, we have decided on the following:
    • We will not be conducting interviews with Polish Wikipedia admins, current or former, due to their late addition to the research plan.
    • We will close all interview collection by Dec 1 (functionally, by Nov 27 due to the US Thanksgiving holiday). This almost certainly means that we will not be able to cover the French current admin section.
cwylo changed Due Date from Nov 27 2024, 6:00 AM to Dec 6 2024, 6:00 AM.Nov 21 2024, 3:11 PM

Thanks for the update @cwylo , just want to flag that our deadline on reporting is firm, which is the reason we aimed for data collection to be complete by Dec 1. So, if you expect interviews to extend into the first days of Dec, it would be good to confirm that we can already have some basic data analysis underway concurrently at that same time so we don't missing our reporting deadline (i.e., waiting until Dec 6 for analysis and reporting to begin has the potential to put the work at risk.)

cwylo changed Due Date from Dec 6 2024, 6:00 AM to Dec 1 2024, 6:00 AM.Nov 21 2024, 7:13 PM

Updates:

  • We will be closing out interview data collection at the end of this week. In total, we have completed 12 participant interviews, 7 with former admins and 5 with current admins. We covered five wikis - en, es, fr, id and ru.
  • Thanks to Bethany, we are obtaining automated transcripts for the few recordings where we didn't have them previously
  • We have begun data analysis, segmenting out themes and key points
  • We have also begun anonymizing participants (using participant numbers) and are adjusting our internal documentation accordingly

Update:

  • Significant and major progress on analysis and report writing. My immeasurable gratitude and respect for everybody working on this project.

I am leaving this ticket open to reflect additional work that needs to be done in Jan '25 to finalize the report and make this work publicly available.

@cwylo thank you so much for your work and this weekly update. We're closing this hypothesis work today and given that the task scoped here is done, I'm going to resolve the task (@Easikingarmager or Claudia: feel free to re-open if work from this task is remaining). We will have a dedicate space for the tail end of the work in Q3 that we will scope and capture. Let's capture your work as part of that task once we create it. thanks.