[Epic] Metrics for sub-referencing
Open, Needs TriagePublic
Actions

Assigned To

None

Authored By

	lilients_WMDE
	Jun 12 2025, 1:37 PM

Description

We want to collect and evaluate data to decide if we reached these annual goals:

The number of human-performed edits adding at least one sub-reference reaches 2000 on dewiki. (Baseline: 0)
The number of references which are similar to another on the same page and not using sub-referencing decreases year over year by 10% on dewiki. Baseline to be set at date of deployment (status quo in June 2024: 6.04% of all references (in numbers 913.175).

We will collect data via scraping data dumps and via user tracking in VE.

Items to complete this epic:

See internal doc for more details.

Related Objects
Search...

Status	Subtype	Assigned	Task
Open		None	T396752 [Epic] Metrics for sub-referencing
Open		None	T398277 [Epic] Scraping data dumps and scraper enhancements
Open		None	T396730 Scraper: Run the scraper with the new features regularly
Resolved		None	T396729 Scraper: Add new metrics for sub-ref data
Resolved		None	T409944 [priority] Scraper: count number of sub-references (on dewiki)
Resolved		None	T410251 Run scraper on dewiki, to count number of sub-references
Resolved	Spike	None	T396720 Scraper: Use Enterprise API to retrieve dumps
Resolved		awight	T409814 Transfer ownership of Elixir MediaWiki library to WMDE Tech Wishes
Open	Spike	None	T410142 Implement Enterprise auth token refresh logic
Open		None	T403298 Provide auth-less access to Enterprise APIs from WMF Analytics cluster
Resolved		None	T410520 Wire http proxy environment vars to scraper web adapter
Open		None	T396731 Scraper: Build a new dashboard for the updated scraper data
Open		None	T410719 Persist scraper outputs to Hive
Resolved		None	T412227 Rate limit EventGate output to 500 events/second
Open		None	T413567 Discussion: scraper outputs vs. event sanitization
Open		None	T412019 [Epic] Schedule scraper and aggregations as an Airflow job
Resolved		awight	T413954 Set up Hive aggregation table and populate with sample data
Resolved		awight	T414803 Write an Airflow sensor to detect Enterprise Snapshots
Resolved		Gehel	T415667 Airflow devenv BashOperator image is lacking libssl1.1
Resolved		awight	T414804 Package scraper binary for Airflow job
Invalid		None	T414809 Write an Airflow sensor for scraper page summaries being fully imported to Hive
Resolved		Tobi_WMDE_SW	T416001 Scraper should write directly to Hive
Open		None	T417370 Cleanup: Elixir Hive adapter should generate better type structure
Resolved		awight	T417633 Airflow devenv (WMDE) cannot see webproxy
Declined		None	T417997 Run Spark Connect server in Analytics cluster
Resolved		None	T418442 Finish and deploy scraper Airflow job
Invalid		awight	T419654 Append environment for BashSensor
Open		None	T419655 Migrate Tech Wishes scraper to gitlab.wikimedia.org
Open		None	T413971 Create a Superset "dataset" and basic chart exposing the scraper per-wiki results
Duplicate		WMDE-Fisch	T414057 Deduplicate page data that got logged multiple times during one run
Resolved		awight	T402613 Production monitoring for TechWishes scraper
Resolved		None	T412018 Make scraper robust against EventGate errors
Declined		None	T412082 Implement resumability for the scraper
Resolved		Tobi_WMDE_SW	T413757 Scraper: Run the "legacy" scraper for dewiki
Resolved		Tobi_WMDE_SW	T416303 Scraper: Run the scraper for all wikis with the January data
Resolved		Tobi_WMDE_SW	T418082 Scraper will output to a simple ND-JSON file
Resolved		None	T418864 Backfill per-wiki scraper summaries into Hive
Open		None	T398435 [Epic] User Tracking in VE for sub-referencing
Resolved		None	T396734 Add more simple logging for subref related events
Open		None	T396735 Aggregate success/abort metrics for subrefs in Superset
Resolved		None	T401494 Add simple logging for the usage of the Convert button
Resolved		None	T401495 Add logging for the subref removals
Resolved		None	T401512 Validate new logging in Superset
Open		None	T403601 Create dashboard to visualize contextItem open vs outcome
Resolved		None	T403720 Add logging for interactions with the subref tooltips and help
Resolved		thiemowmde	T403693 Implement tooltips onboarding for subreferencs
Resolved		None	T403694 Add help link to the edit panel when editing details
Open		None	T397124 Try to categorize different subref usage types