Page MenuHomePhabricator

[Analytics] [Request] [WDQM] Devise Global Wikidata Quality Metric MVP
Open, Needs TriagePublic

Description

WMDE Analytics Request

This task was generated using the WMDE Analytics request form. Please use the task templates linked on our project page to create tasks for the team. Thank you!

Why (Context & Decision)

What problem are you trying to solve, and what decision will this analysis inform? Briefly explain the organisational or strategic context, why this matters now, and what action will be taken based on the outcome.

As a part of T427512: [EPIC] [WDQM] Wikidata Quality Metric, we want to create v0.1.0 of the Wikidata Quality Metric including the first two dimensions from T427513 and T427514.

What (Scope & Output)

Describe the specific question(s), metrics, segments, or deliverables (e.g., dashboard, deep dive, experiment analysis), including any relevant definitions or constraints.

  • Create MVP metric on Superset:wd_quality_metrics
  • Run the process over the weeks in June
    • Only available data was from May when T427513 and T427514 were being worked on and there's a manual step to estimate mandatory constraint violations
  • Discussion of current dimension metrics created in T427513 and T427514
  • Looking into how much these metrics change over a few months and deciding on a multiplier to create a metric that's not between 0 and 1
  • Discussion of and decision on the fractional components of the full Wikidata Quality Metric for the currently completed dimensions
  • Document the decisions

By When (Timing & Priority)

Provide a clear deadline, any key milestones (e.g. launch, leadership review), and note if timing is flexible or fixed.

30.06.2026


Information below this point is filled out by the analyst.

Sub Tasks

A breakdown of the steps to complete this task.

See outputs above

Estimation

Estimate:
Actual:

Data

The tables that will be referenced in this task and the sample sizes from them that will be used.

Event Timeline

AndrewTavis_WMDE renamed this task from [Analytics] [Request] [WDQM] Devise Global Wikidata Quality Metric to [Analytics] [Request] [WDQM] Devise Global Wikidata Quality Metric MVP.May 28 2026, 3:14 PM
AndrewTavis_WMDE updated the task description. (Show Details)

I'll pick up work on this when I'm back :) A few points:

  • I'll need to fine tune the estimation of mandatory constraint consistency
  • The underlying dataset is new, so we only have data for May and can't compare more than one month
  • It'd be good to look into getting expanded access to the Action API so I can check 500 claim IDs per call instead of just 50
    • With this we can expand the sample from the 10-20,000 that has been used till now