Page MenuHomePhabricator

[Q3 FY 24-25 Applied Sciences] Survey Research
Closed, ResolvedPublic

Description

This is a parent task to capture the Q3 work by Applied Sciences (Research) related to Surveys. It will capture prioritization decisions and major weekly updates related to tasks in this bucket from January - March 2025. More fine-grained updates and coordination with occur in the subtasks as appropriate.

Confirmed Projects

ProjectResponsiblePrioritizationDetails (if applicable)Final status
Moderator Tools Support@YLiou_WMFOKRT377264Ongoing
Survey Reflection Report@TAndicEssential WorkT382890Complete and passed to @Isaac for review
Developer Satisfaction Survey (DSS)@CMyrick-WMFEssential WorkT382718Complete
Reader Survey@YLiou_WMFEssential WorkT382611Ongoing as shift from analysis -> reporting
Community Insights publishing@TAndicEssential WorkT369216Complete
Safety Survey Report@TAndicEssential Workm:Community SafetyComplete
Survey Support Desk@TAndicEssential Work(past reports: T377630)Ongoing
Wikipedia Library Support@YLiou_WMFEssential WorkAsana (private)Ongoing

Event Timeline

Isaac renamed this task from [Q3 FY 24-25] Survey Research to [Q3 FY 24-25] Applied Sciences Survey Research.Jan 13 2025, 7:22 PM
Isaac renamed this task from [Q3 FY 24-25] Applied Sciences Survey Research to [Q3 FY 24-25 Applied Sciences] Survey Research.
Isaac triaged this task as High priority.Jan 13 2025, 11:01 PM

Update (Jan 17, 2025):

Applied Sciences provided input into the Future Audiences Survey & Metrics Insights Update to the Foundation Metrics working group:

  • Slides summarizing insights on contributors drawn from the Community Insights surveys provided by @TAndic
  • Slides summarizing insights on readers drawn from the Readers surveys provided by @YLiou_WMF
  • These are intended to inform the Foundation Annual Planning Process, specifically by helping to answer the following questions:
    1. Who is visiting Wikipedia today (and who is not)?
    2. Who is volunteering their time to contribute to Wikipedia? How is this changing?
    3. What other platforms do people use & trust for information?

Readers Survey 2024:

  • Survey data analysis continues and weighting/de-biasing process to begin next week, but no major updates

Community Insights Survey 2024:

  • @TAndic presented initial draft report to Applied Science team on Monday for feedback / input.

Developer Satisfaction Survey 2024:

  • @CMyrick-WMF has completed preliminary visualizations and will share with stakeholders by early next week for feedback / requests for additional analysis

Update (January 24, 2025):

Readers Survey 2024:

  • No major updates as analysis and visualization is underway

Community Insights Survey 2024:

Survey Reflection Report:

  • @TAndic and @Bethany kicked off collaboration on Survey Reflection Report

Developer Satisfaction Survey:

  • @CMyrick-WMF provided cleaned results data to stakeholders for exploration
  • stakeholders continued to view the visualizations, give feedback, and make requests

Update (January 31, 2025):

Readers Survey 2024:

  • Started process with @Isaac to begin de-biasing data
  • Began updating meta-wiki page with information on survey administration
  • Continuing visualization (to be revised once data are weighted)

Community Insights Survey 2024:

  • @TAndic participated in the sprinthackular with Design on prototyping a QuickSurveys UI (internal-only deck)
    • provided feedback on appropriate guardrails for internal / community use for any QuickSurveys updates
  • @TAndic Answered questions from community members on Community Insights

Update (February 7, 2025):

Moderator Tools Satisfaction Surveys:

  • Patroller Tools baseline satisfaction survey script approved and shared with @Bethany for translation.
    • We chose to go through the Movement Comms translation process and as such, will only proceed with translations to arabic, german, french, indonesian, hungarian, and urdu
  • Next steps are survey programming in Limesurvey (initially in English) and testing

Readers Survey 2024:

  • No major updates, analysis and visualization continuing

Developer Satisfaction Survey 2024:

  • Requested visualizations and crosstabs complete
  • Text summaries accompanying each visualization (to be included in public report) added

Community Insights Survey:

  • Community Insights metawiki page and inter-linking updated with link to report and plans for the next survey (may have questions from community members over the next couple of weeks)
  • @TAndic met with Investigating Neurodivergent Wikimedian Experiences researchers to discuss Community Insights data and surveying about disability
  • Finalized V2 archetypes work by sharing descriptives for the slide deck tables

Survey Reflection Report:

  • No major updates this week

Safety Survey:

  • Committee Support team shared preliminary safety survey results (from Q2) to the Universal Code of Conduct Committee

Update (February 14, 2025):

Moderator Tools Satisfaction Surveys:

  • Translations should be completed by EOW
  • Survey programming to begin next week

Developer Satisfaction Survey 2024:

  • Finalized visualizations with file names and descriptions uploaded (~90 in total) to Commons using pywikibot

Community Insights Survey

  • Report announced to staff, epic ticket ready for closing

Survey Reflections Report

  • First draft shared to @Isaac for feedback

Safety survey:

  • @TAndic regrouped with stakeholders and movement comms on how to best report data, will work with Movement Comms over the next few weeks

Readers Survey 2024:

  • Hit some IT snags with setting up SSH access and JupyterHub access, but able to start compiling comparison data for weighting / debiasing
  • Basic preliminary (unweighted) visualization code is complete and ready for weighted data

Update (February 21, 2025):

Moderator Tools Satisfaction Surveys

  • Translations completed

Developer Satisfaction Survey 2024:

  • no updates this week

Survey Reflections Report:

  • Began incorporating feedback into the document

Safety Survey:

  • Combined newest data with 2023 data to enable analysis of change over time
  • Existing deck shared to ArbCom by Committee Support

Readers Survey 2024:

  • No updates this week

Update (February 28, 2025):

No major project updates this week!

Planning for future survey research is underway and the broader "survey team" will meet next week to generate ideas as part of APP.

Update (March 7, 2025):

Moderator Tools Satisfaction Surveys:

  • RecentChanges/Watchlist baseline surveys have been translated, programmed, and shared with colleagues for testing.
  • Surveys will be fielded via QuickSurvey to logged-in editors with 500+ edits and accounts at least 6 months old (who have not opted out of QS)
    • Coverage ratios are still TBD, but we anticipate fielding next week
  • @YLiou_WMF consulted with Moderator Tools team, @CMyrick-WMF, and Daisy C on best approach for research recruitment for Watchlist prioritization project

Developers Satisfaction Survey 2024. @CMyrick-WMF:

  • Provided guidance for, and worked with section leads on, qualitative analysis of free-response questions
  • Helped move report content into on-wiki sandbox
  • Determined weighting for Gerrit-user survey respondents' responses, by comparing actual Gerrit commits from last year (based on @thcipriani's query of actual Gerrit commits), with self-reported Gerrit commit number ranges given by survey respondents
  • Reran Gerrit-related visualizations using new weights

Survey Reflection Project:

  • @TAndic Draft (with input from @Bethany, @CMyrick-WMF, Daisy C, Mike R, and @YLiou_WMF) submitted to @Isaac for final feedback and decision-making.
  • First draft of Codex-aligned LimeSurvey template is live and useable thanks to @Bethany and Daniel (being used for Moderator tools surveys)
    • Discussion of further fixes (eg, arbitrary word-wrapping issue) to take place next week.

Safety Survey:

  • @TAndic is drafting tables mockup for metawiki reporting with Movement Comms; as Charts extension will go into implementation later in the year we will publish the tables this month with current and most recent previous wave for comparison, and then implement charts later when they're available

Survey Support:

  • @TAndic responded to CentralNotice discussion on survey deployments / new process
  • @TAndic provided feedback on Research grantee survey (in consultation with @YLiou_WMF)

Preliminary planning doc for future survey research shared with @leila (thanks to @TAndic, @CMyrick-WMF, @Bethany, Mike R, and @Isaac for their invaluable input)

Update (March 14, 2025):

Moderator Tools Baseline surveys:

  • Surveys have been updated based on feedback and are ready to field
  • QS coverage ratios are still being determined
  • Surveys should be ready to field next week

Developers Satisfaction Survey:

    • Qualitative analysis of free-response questions has been incorporated into report draft
  • Final review of report draft is underway with stakeholders
  • Public versions of DSS scripts have been uploaded to this Gitlab repo

Survey reflection/implementation:

  • Continuing on LimeSurvey guides, shared the first ones to a new staff LimeSurvey user for feedback on whether they're helpful as they onboard

Safety Survey:

  • Results are published! We may update these with charts when the Charts extension is running again, but this table format of reporting should allow us to just add a line to each project when new results are ready, rather than having to update analysis text and re-translate each wave. The results will be linked to in a Diff post planned by T&S+Movement Comms next month.

Survey support:

  • @TAndic Assisted Committee Support team with Qualtrics setup for a more complex survey

Readers Survey:

  • Preliminary weighted results to be shared during Applied Sciences meeting on March 24

Update (March 21, 2025):

Moderator Tools Baseline surveys:

  • Decision made to field surveys in smaller projects at 100% coverage
  • Based on response rate, we will adjust planned coverage rate for enwiki, commons, wikidata, dewiki, and frwiki

Developers Satisfaction Survey:

  • Mediawiki report to be published today (March 21)
  • Slide decks being prepared to share during upcoming P&T department staff meeting

Survey Support:

Readers Survey:

  • Slide deck being prepped to share on Monday!

Update (March 28, 2025)

Moderator Tools Baseline surveys:

  • Patroller tools survey will field as soon as privacy statement is finalized

Developers Satisfaction Survey:

Survey Reflection Report:

  • Final document passed on to Isaac for last check, task closing end-of-day

Survey Support:

  • Assisted with final testing on Committee Support survey, they aim to launch next week.

Contributor Metrics Strategy (update here as it's not covered by other Phab tasks):

  • Reviewed latest draft of contributor metrics decision brief; this work will bridge slightly to next quarter for a finalization meeting due to group scheduling conflicts at the end of this quarter/beginning of next quarter. Anticipated close in mid-to-late April.

Readers Survey:

  • Results presented to Applied Science team for feedback and questions
  • Results are being prepared to be shared initially on Officewiki and will be posted to Meta-wiki next week pending feedback from the Survey Alignment Working Group (Future Audiences, Research, Audience Insights)
Isaac updated the task description. (Show Details)

Thanks @YLiou_WMF !