Page MenuHomePhabricator

[Q4 FY 24-25 Applied Science] Knowledge Gaps Research
Closed, ResolvedPublic

Description

This is a parent task to capture the Q4 work by Applied Sciences (Research) related to Knowledge Gaps. It will capture prioritization decisions and major weekly updates related to tasks in this bucket from April - June 2025. More fine-grained updates and coordination will occur in the subtasks as appropriate. It follows the Q3 task (T383614).

Confirmed Projects

ProjectResponsiblePrioritizationTicketStatus
Readers survey analysis@YLiou_WMFEssential WorkT382611Complete
Youth Conference@YLiou_WMFEssential WorkN/AComplete
Developer satisfaction share-out@CMyrick-WMFEssential WorkT391146Complete
Editor metric consultation@TAndicOKR consultationT369096Complete
Language vision exploration@CMyrick-WMFEssential WorkN/AOngoing
Topic Infrastructure@IsaacTBDT361637Ongoing
Language switching baseline analysis@CMyrick-WMFEssential WorkmetaComplete

Details

Due Date
Jun 30 2025, 4:00 AM

Event Timeline

Isaac triaged this task as High priority.Apr 11 2025, 5:31 PM
Isaac updated the task description. (Show Details)
Isaac set Due Date to Jun 30 2025, 4:00 AM.

Weekly update:

  • Developer satisfaction share-out: @CMyrick-WMF shared DSS 2025 highlights with Product & Tech. Closed out T391146 and its epic.
  • Editor metric consultation: @TAndic met with working group to decide on uncertain measurements and reach consensus on recommended metrics. Decision brief is now in the final stages (executive summary developed, awaiting sign-off) - this work should now wind down for the Research team.

Weekly update:

  • Youth Conference: @YLiou_WMF is preparing a 10-minute presentation on Youth Participation that will draw primarily on Community Insights and the Global Readers survey, with a little bit from the Admin Research surveys

Weekly update:

  • Editor metric consultation: @TAndic decided on recommended thresholds for active experienced editors
  • Youth Conference: @YLiou_WMF presenting this week
  • Language vision exploration: @CMyrick-WMF finished first draft of vision work, and now working through comments from @Isaac

Weekly update:

  • Editor metric consultation: In investigating edit attempts to assess a potential metric for retained newcomers, @Hghani found that the table is incomplete, seemingly primarily affecting mobile edits. Phab ticket filed here: T394961

Note: I added @CMyrick-WMF's project on language switching analysis. This is a sprint to understand what data we have about the degree to which readers switch language editions while reading Wikipedia. It will culminate with Caroline updating my former meta page on the topic with new details on the available datasets, high-level descriptive statistics, and how they differ. It comes out of T391112 but we are keeping it as a separate project from that overall reader discovery work. No Phabricator task given the short nature but we will wrap it up this quarter (by June 30th) before considering the next steps in this space.

Weekly update:

  • Youth Conference
    • Presentation was well-received and attracted a lot of interest from attendees
    • @YLiou_WMF has begun corresponding with a couple of attendees who have had some follow-up questions.
  • Readers survey analysis: began an exploratory segmentation analysis based on reader motivations.
  • Language switching baseline analysis
    • @CMyrick-WMF continued exploration of language-switching data using
      • wmf.pageview_actors (a subset of wmf.webrequest)
      • wmf.interlanguage_navigation (counts of web requests which stay within a project family but change language)
      • event.universallanguageselector (a log of universal language selector usage)
    • Documented learnings in metawiki sandbox (to be moved to Isaac's former meta page)

Weekly update

Language switching baseline analysis:

  • Continued exploration of data, comparing the use of different tables
  • Began calculations of baselines:
    • % of daily readers who switch languages
      • initial findings suggest the daily percentage may be around 5%
      • planning to look at %s when data disaggregated by access_method, loggedIn, etc.
    • # of languages readers (who read 2+ languages in one day) read
      • vast majority read 2 languages
      • those in the 99th percentile read 4 or fewer
    • Will share notebook or trends/initial findings next week

Weekly update:

  • Editor metric consultation: Consultation complete 🎉, the group has aligned on a proposal for retained newcomers
  • Readers survey: Readers Survey published on meta-wiki 🎉
  • Topic Infrastructure: We had a discussion with several technical/Product stakeholders about how to bring our proposed topic model improvements to Production and are drafting a proposal for how to move this project forward.

Weekly update

Language switching baseline analysis:

  • Continued exploration of data, comparing the use of different tables
    • Determined that, because the Universal Language Selector is a desktop-only feature; we will likely not use the UniversalLanguageSelector table for the current study since it does not capture language switches on mobile web or mobile app.
  • Continued calculations of baselines (with a focus on methodological considerations):
    • Regarding the % of readers who switch languages at least once during the day: the previous estimated only included pageviews-from-internal-referals in the denominator. We should also look at the % using a denominator that includes all Wikipedia pageviews (internal and external referers).
    • The most common language being translated from and to is English. However, English Wikipedia also has the highest readership overall; so we need to normalize based on total pageviews. Normalization is in progress.
    • Desktop language-switches and mobile web language-switches make up similar %s of daily switches; mobile app language-switches make up a much lower %s of daily switches. However, desktop and mobile readership looks different across wikis, so we need to normalize based on each Wikipedia's overall desktop pageview trends and mobile web pageview trends.
Isaac updated the task description. (Show Details)