Write data to Iceberg formatted tables (mediawiki.page_content_change)
Closed, DuplicatePublic5 Estimated Story Points
Actions

Assigned To

None

Authored By

	lbowmaker
	May 16 2023, 12:19 AM

Description

User Story

As a data engineer, I need to write the parsed content from mediawiki.page_content_change to the tables designed and implemented in https://phabricator.wikimedia.org/T336714

Done is:

Data is written to tables as implemented in https://phabricator.wikimedia.org/T336714

Related Objects
Search...

Status	Assigned	Task
Invalid	VirginiaPoundstone	T345988 [Epic] XML MediaWiki data dumps for right to fork
Resolved	Milimetric	T330296 Dumps 2.0 Phase I: Proof of concept for MediaWiki XML content dump via Event Platform, Iceberg and Spark
Resolved	Milimetric	T335859 Document Solutions and Decisions for Realtime Dump Implementation
Resolved	xcollazo	T336714 Design and implement tables to store parsed content from mediawiki.page_content_change
Resolved	xcollazo	T335860 Implement job to transform mediawiki.page_content_change
Duplicate	None	T336718 Write data to Iceberg formatted tables (mediawiki.page_content_change)

Event Timeline

lbowmaker created this task.May 16 2023, 12:19 AM

lbowmaker set the point value for this task to 5.

Antoine_Quhen claimed this task.May 25 2023, 3:27 PM

Antoine_Quhen removed Antoine_Quhen as the assignee of this task.

Antoine_Quhen moved this task from Next Up to In Progress on the Data Pipelines (Sprint 14) board.

Antoine_Quhen moved this task from In Progress to Next Up on the Data Pipelines (Sprint 14) board.

Antoine_Quhen subscribed.

Antoine_Quhen renamed this task from Write data to Iceberg formatted tables to Write data to Iceberg formatted tables (mediawiki.page_content_change).May 25 2023, 4:06 PM

This does indeed seem to be a duplicate of T335860: Implement job to transform mediawiki.page_content_change, but I won't merge them until others can have a look.

JArguello-WMF added a project: Data Engineering and Event Platform Team.Jun 30 2023, 4:44 PM

JArguello-WMF moved this task from Data Eng Backlog to Sprint 0 on the Data Engineering and Event Platform Team board.Jun 30 2023, 4:46 PM

JArguello-WMF edited projects, added Data Engineering and Event Platform Team (Sprint 0); removed Data Engineering and Event Platform Team.

Sounds like a dup to me.

lbowmaker moved this task from Sprint 0 to Sprint 1 on the Data Engineering and Event Platform Team board.Jul 25 2023, 12:19 PM

lbowmaker edited projects, added Data Engineering and Event Platform Team (Sprint 1); removed Data Engineering and Event Platform Team (Sprint 0).

xcollazo closed this task as a duplicate of T335860: Implement job to transform mediawiki.page_content_change.Jul 26 2023, 3:08 PM

Write data to Iceberg formatted tables (mediawiki.page_content_change)Closed, DuplicatePublic5 Estimated Story PointsActions