[Epic] Metrics for sub-referencing
Open, Needs TriagePublic
Actions

Assigned To

None

Authored By

	lilients_WMDE
	Jun 12 2025, 1:37 PM

Description

We want to collect and evaluate data to decide if we reached these annual goals:

The number of human-performed edits adding at least one sub-reference reaches 2000 on dewiki. (Baseline: 0)
The number of references which are similar to another on the same page and not using sub-referencing decreases year over year by 10% on dewiki. Baseline to be set at date of deployment (status quo in June 2024: 6.04% of all references (in numbers 913.175).

We will collect data via scraping data dumps and via user tracking in VE.

Items to complete this epic:

See internal doc for more details.

Related Objects
Search...

Status	Subtype	Assigned	Task
Open		None	T396752 [Epic] Metrics for sub-referencing
Open		None	T398277 [Epic] Scraping data dumps and scraper enhancements
Open		None	T396730 Scraper: Run the scraper with the new features regularly
Resolved		None	T396729 Scraper: Add new metrics for sub-ref data
Resolved		None	T409944 [priority] Scraper: count number of sub-references (on dewiki)
Resolved		None	T410251 Run scraper on dewiki, to count number of sub-references
Resolved	Spike	None	T396720 Scraper: Use Enterprise API to retrieve dumps
Resolved		awight	T409814 Transfer ownership of Elixir MediaWiki library to WMDE Tech Wishes
Open	Spike	None	T410142 Implement Enterprise auth token refresh logic
Open		None	T403298 Provide auth-less access to Enterprise APIs from WMF Analytics cluster
Resolved		None	T410520 Wire http proxy environment vars to scraper web adapter
Open		None	T396731 Scraper: Build a new dashboard for the updated scraper data
Open		None	T410719 Persist scraper outputs to Hive
Resolved		None	T412227 Rate limit EventGate output to 500 events/second
Open		None	T413567 Discussion: scraper outputs vs. event sanitization
Open		None	T412019 [Epic] Schedule scraper and aggregations as an Airflow job
Open		WMDE-Fisch	T413954 Set up Hive aggregation table and populate with sample data
Open		None	T414803 Write an Airflow sensor to detect Enterprise Snapshots
Open		awight	T414804 Produce conda-dist package for scraper
Open		None	T414809 Write an Airflow sensor for scraper page summaries being fully imported to Hive
Open		None	T413971 Create a Superset "dataset" exposing the scraper per-wiki results
Duplicate		WMDE-Fisch	T414057 Deduplicate page data that got logged multiple times during one run
Open		None	T402613 Production monitoring for TechWishes scraper
Resolved		None	T412018 Make scraper robust against EventGate errors
Declined		None	T412082 Implement resumability for the scraper
Open		None	T413757 Scraper: Run the "legacy" scraper for dewiki
Open		None	T398435 [Epic] User Tracking in VE for sub-referencing
Resolved		None	T396734 Add more simple logging for subref related events
Open		None	T396735 Aggregate success/abort metrics for subrefs in Superset
Resolved		None	T401494 Add simple logging for the usage of the Convert button
Resolved		None	T401495 Add logging for the subref removals
Resolved		None	T401512 Validate new logging in Superset
Open		None	T403601 Create dashboard to visualize contextItem open vs outcome
Resolved		None	T403720 Add logging for interactions with the subref tooltips and help
Resolved		thiemowmde	T403693 Implement tooltips onboarding for subreferencs
Resolved		None	T403694 Add help link to the edit panel when editing details
Open		None	T397124 Try to categorize different subref usage types