Page MenuHomePhabricator

Define Metrics for Survey-Based Knowledge Gaps
Open, Needs TriagePublic

Description

Out of the gaps in the Knowledge Gaps Taxonomy, define metrics for the survey-based gaps, namely those gaps whose measurement depend on surveys, as identified in the metrics spreadsheet.

This entails 2 main tasks:

  • Gather survey questions for gaps that we want to measure.
  • Define metrics reflecting inequality in the distribution over the gap categories.

Timeline: End of FY21

Event Timeline

Weekly updates:
gathered survey questions from various surveys used across different initiatives at the Foundation, mapped to the knowledge gaps in the taxonomy and collected them here: https://docs.google.com/spreadsheets/d/1xZCOSyoZXr9oVTqZRPmR47Rq_gHDYNZuvH-XFySFzC4/edit#gid=0
Technical skills is still missing but @leila mentioned that these questions have been asked as part of previous research.

@Miriam thank you for the update.

Re the Technical skills questions: what we have used in the past is here (section II). Admittedly, the question is based on a 2012 study by Hargittai et al. and some refresh may be needed given the 9 years gap in between. I suggest you put those as fall back option. I'll write to a couple of people who may know more.

Weekly updates:

  • Started exploring ideas for metrics, met with @marcmiquel to hear about community perspectives

Weekly updates:

  • Worked on a more generic framework for metrics deliverable from all dimensions

Weekly updates:

  • Defined a few questions based on which we will design metrics. These are questions users might want to ask the index, and that reflect different aspects of the quantification of a knowledge gap. These questions can be answered by extracting different values from the distribution of data over the categories of a gap.

Weekly updates

  • Started working on extracting features from a real distribution (articles by number of images), based on the metrics questions we are defining. This is to understand the extent to which these metrics are interpretable and understandable by non-technical people.

Weekly updates:

  • With the help of @marcmiquel , we are figuring out the details of the questions for metrics deployment. I will start computing those on toy data coming from the cultural observatory.

Weekly updates:

  • Preparing a presentation to summarize the efforts on this front to Tech Managers
  • Working on plotting and rendering some of the metrics from tagged content