Pageviews ingestion Investigation [10% time]
Open, LowPublic
Actions

Assigned To

None

Authored By

	FNavas-foundation
	May 15 2025, 3:32 PM

Description

As a WME Engineer i want to document an approach to fetch data from WMF datalake to our side.

We are building pageviews dataset, on top of existing data, on wmf side. This is to simplify the ingestion and getting close to a near realtime batch processing. Instead of having to download hourly batches we looking into have a more flexible approach.

The goal here is just a document to have internal discussion and also discussions with Data Platform Team.

Acceptance Criteria

Documented approach
Presented to team

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Open		None	T355903 Pageviews {Content Integrity}
		Open		None	T394428 Pageviews ingestion Investigation [10% time]

Event Timeline

FNavas-foundation created this task.May 15 2025, 3:32 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 15 2025, 3:32 PM

FNavas-foundation moved this task from Incoming to To Be Estimated/To Be Discussed- Migrating on the Wikimedia Enterprise board.May 15 2025, 4:40 PM

JArguello-WMF moved this task from To Be Estimated/To Be Discussed- Migrating to Content Integrity PB- Migrating on the Wikimedia Enterprise board.May 23 2025, 4:32 PM

FNavas-foundation renamed this task from [stub] pageviews infra ideas ricardo to Prep infra ideas for receiving pageviews from wmf.May 28 2025, 5:51 PM

FNavas-foundation updated the task description. (Show Details)

FNavas-foundation moved this task from Content Integrity PB- Migrating to To Be Estimated/To Be Discussed- Migrating on the Wikimedia Enterprise board.May 28 2025, 5:55 PM

• REsquito-WMF renamed this task from Prep infra ideas for receiving pageviews from wmf to Pageviews ingestion Investigation.May 29 2025, 11:29 AM

• REsquito-WMF triaged this task as Low priority.

• REsquito-WMF updated the task description. (Show Details)

JArguello-WMF moved this task from To Be Estimated/To Be Discussed- Migrating to Sprint 78 on the Wikimedia Enterprise board.May 29 2025, 2:33 PM

JArguello-WMF edited projects, added Wikimedia Enterprise (Sprint 78); removed Wikimedia Enterprise.

• REsquito-WMF renamed this task from Pageviews ingestion Investigation to Pageviews ingestion Investigation [10% time].May 29 2025, 2:34 PM

JArguello-WMF reassigned this task from • REsquito-WMF to LDlulisa-WMF.May 30 2025, 5:12 PM

JArguello-WMF added a subscriber: • REsquito-WMF.

JArguello-WMF moved this task from Backlog to Concept Development on the Wikimedia Enterprise - Content Integrity board.Jun 4 2025, 12:19 PM

JArguello-WMF moved this task from Next Up to Blocked/Paused on the Wikimedia Enterprise (Sprint 78) board.Jun 4 2025, 1:25 PM