Page MenuHomePhabricator

[Q1 FY 25-26 Applied Sciences Team] Building the Foundations Research
Closed, ResolvedPublic

Description

This is a parent task to capture the Q1 work by Applied Sciences (Research) related to Building the Foundations. It will capture prioritization decisions and major weekly updates related to tasks in this bucket from July - September 2025. More fine-grained updates and coordination will occur in the subtasks as appropriate. It follows the Q4 task (T391719).

Confirmed Projects

ProjectResponsiblePrioritizationTicket
Reader Foundational@YLiou_WMFOKRT398379
Understanding mentorship experiences@TAndicOKRT398250
AI use-cases write-up@diegoEssential WorkT382727
Retraining recommendation@diegoEssential WorkT399726
Survey support desk@TAndicEssential WorkT400691
Research to ML code migration@fkaelinEssential WorkT398974
Reader Research Direction@MGerlachEssential WorkT400030

Details

Due Date
Sep 30 2025, 4:00 AM

Event Timeline

Isaac set Due Date to Sep 30 2025, 4:00 AM.
Isaac added a subscriber: fkaelin.

Update July 18, 2025:

  • Reader Foundational research: v2 of the OE use cases survey will be fielded starting July 23, 2025; analysis of v1 responses and reading data collection is underway
  • Survey support desk: @YLiou_WMF provided consultative support for the Wikimania 2025 after-event survey

Update July 25, 2025:

  • Reader foundational research: v2 survey was fielded starting July 24; analysis of v1 survey results continues; preliminary analysis of v2 results and drafting of phase 2 survey has begun.

Update August 1, 2025:

  • Reader foundational research: v2 survey responses have been team-coded; analysis of responses and session data continues. Preliminary framework based on OE surveys to be shared with stakeholders next week.
  • Survey support desk: discovered an issue with the SMTP email setup for Qualtrics, working with SRE to get it fixed for the upcoming Readers Diary study
  • Understanding mentorship experiences: stakeholder requests & suggestions in the newcomers survey draft are being incorporated into a revised questionnaire; qualitatively explored a small random sample of editors who fit the survey target population to better understand what kinds of questions make sense to ask and what sorts of patterns of behavior and dynamics the larger group might have
  • AI use cases write-up:
    • Discussing with @Miriam where to publish this work.

Weekly updates:

  • Reader foundational research (T398379): (from @MRaishWMF while @YLiou_WMF is away!)
    • continued developing a framework that describes use cases in terms of on-site behavior using examples from recent data collection to illustrate it
    • presented early findings to some stakeholders for feedback on project direction
    • continued working to develop the direction of phase 2 (diary study)
  • Retraining recommendation (T399726):
    • Build new evaluation dataset
    • Compare different retraining period lengths (3 months, vs 1 year)
  • Research to ML code migration (T398974):
    • Initial discussion for scoping with Özge. The proposal will be based on two components: a project in ml-pipelines for shared code, and the usage of the Command api for airflow configuration.
  • Understanding mentorship experiences (T398250):
    • Programmed the first draft of the newcomers survey in LimeSurvey (feel welcome to test it and give feedback on the ticket!)
Isaac added a subscriber: MGerlach.

Weekly update:
Reader foundational research:

  • incorporating stakeholder feedback on project direction and use case framework

Survey support desk:

  • Yu-Ming and Tanja inputted to new QuickSurveys infrastructure improvements requests version based on recent experiences with surveys and identified needs within the Foundation

Understanding mentorship experiences:

  • Gathered feedback from stakeholder and Research team testers of the survey draft, made a codebook for the survey with strategic mapping of the questions, began working on a resource end-page for survey respondents.

Retrain recommendation:

  • Main conclusion: Models should be retrained at least once a year. After 1 year models lose 1% precision (details in T399726#11090332)

Weekly update:
Reader foundational research:

  • incorporating stakeholder feedback on project direction and use case framework

Survey support desk:

  • Yu-Ming and Tanja inputted to new QuickSurveys infrastructure improvements requests version based on recent experiences with surveys and identified needs within the Foundation

Understanding mentorship experiences:

  • Gathered feedback from stakeholder and Research team testers of the survey draft, made a codebook for the survey with strategic mapping of the questions, began working on a resource end-page for survey respondents.

Weekly update:

  • Understanding mentorship experiences: Research page published as well as the questionnaire (for community reference and documentation), drafted a Resources page for respondents, and the privacy statement request is in progress. Miriam and Tanja worked on finalizing the KR language and it should be added to the WE1.1 KRs soon.

Weekly update:

  • Reader Foundational Research: we have confirmed logistics for recruitment of Userlytics panelists for the diary study
  • Understanding mentorship experiences: 0-coverage QuickSurvey was deployed and appears to be working well; the survey was announced on the enwiki Village Pump on the 3rd; we're scheduled to increase coverage on the QuickSurvey and start collecting data on the 9th!
  • Reader Research Direction: initial draft has been completed and shared for feedback

Weekly update:

  • Reader Foundational Research:
    • Diary study data collection is underway
    • Work on closed-ended survey instrument draft is underway, research team met to establish timelines for fielding dates in light of expected temp user account introduction on English Wikipedia (currently slated for Early October): specifically, aim to complete data collection before this
  • QuickSurveys: Yu-Ming and Tanja inputted to QuickSurvey updates scoping (working document [internal]).
  • Newcomers survey: The survey launched on the 9th and is collecting data! An issue with ad blockers affecting event logs is causing a proportion of responses to not have a corresponding quicksurvey event; Tanja and Yu-Ming are brainstorming on how to account for this with weights.

Weekly update:

  • Reader Foundational Research:
    • Diary study data collection continues
    • Closed-ended survey draft shared with stakeholders, request for new privacy statement submitted, beginning process for programming and deployment
  • Newcomers survey: we're ready to undeploy the survey next Monday; started working on R cleaning and recode code.
  • Survey support desk (related to Newcomers survey): we've been exploring missing data from quicksurvey initiation and response event logs; we hope to have some understanding of patterns of missing data from this exploration, and may be able to help other teams working on event loss as externally linking surveys are a unique data point.

Resolving as this was the tracker task for Q1