Page MenuHomePhabricator

FY 2023-2024 Reader/Contributor Knowledge Gap Metric Planning
Open, In Progress, Needs TriagePublic

Description

Building on the Knowledge Gap content metrics as well as foundational work by @JAnstee_WMF and @Miriam, (summarized here) @YLiou_WMF will work with @CMyrick-WMF and @TAndic to map out scenarios for readers and contributors knowledge gap metrics roughly following this 2 x 2 based on GDI's prior experience with contributors research and Research's prior experience with readers research:

Ideal / Stretch ScenarioIn Practice / High-Confidence Scenario
Descriptive / What Metrics
"Inferential" / Why Metrics

Event Timeline

YLiou_WMF changed the task status from Open to In Progress.Jun 2 2023, 3:58 PM
YLiou_WMF updated the task description. (Show Details)

Met with MR, CM, and TA on 5/31 for background on Knowledge Gap Index work and to kickoff planning project.

Update:

Met with Caroline and Tanja to discuss readers research ideas (brainstorm tracked here along with previous contributors brainstorm).

Received clarification from Miriam that we should focus on the broadest possible readers sample (essentially replicating previous Readers research conducted by Isaac. Plan is to try to replicate previous research with updates reflecting new demographic questions used in 2022 Community Insights surveys (in particular, the gender (DP03) and minority (DPP01) and discriminated group (DPP02) identity items). Project timeline needs to accommodate design, sampling, fielding, collection, and analysis by the end of CY2023.

Update: Following input from @TAndic and @Isaac, I've developed a preliminary project plan and timeline (very much a work in progress).

Notes:

  • As noted previously, we plan to utilize wherever possible, already-translated items drawn from the 2019 Readers survey (also tracked here) as well as the most-recent (2022) Community Insights survey. Although note that there is considerable non-overlap between these two surveys (see here).
  • De-biasing procedures may be substantially simplified relative to those employed in 2019 and are currently planned to focus on demographic weighting for geography (possible to revisit this as part of analysis in Q3/Q4?)
  • Translation procedure still TBD (Qualtrics as with Community Insights or using Movement Comms process)

@YLiou_WMF Hi, please keep at least one active project tag on a task, so the task can be found on a workboard - thanks!