Maniphest T305843

Migrate the Clickstream jobs to Airflow
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• EChetty
	Apr 11 2022, 1:17 PM

Tags

Referenced Files

None

Subscribers

Description

Goal:
Migrate the Clickstream jobs to Airflow:

Job Details:

Input	Processing	Output
Hive Table	Spark	Archive

Success Criteria:

Have the 1 Monthly Job Migrated (SLA 42 Days)

Related Objects

Mentioned In: T289532: Add more languages to Wikipedia Clickstream

Event Timeline

• EChetty created this task.Apr 11 2022, 1:17 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 11 2022, 1:17 PM

Antoine_Quhen claimed this task.Apr 15 2022, 4:40 PM

Antoine_Quhen moved this task from Discussed (Radar) to Estimated on the Data Pipelines board.

Antoine_Quhen added a project: Data-Engineering-Kanban.

Antoine_Quhen moved this task from Next Up to In Progress on the Data-Engineering-Kanban board.

https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/52

Antoine_Quhen moved this task from Estimated to In Review on the Data Pipelines board.May 9 2022, 3:02 PM

Antoine_Quhen moved this task from In Progress to In Code Review on the Data-Engineering-Kanban board.

Antoine_Quhen moved this task from In Code Review to Done on the Data-Engineering-Kanban board.May 23 2022, 3:05 PM

Antoine_Quhen moved this task from In Review to Done on the Data Pipelines board.May 23 2022, 3:08 PM

JArguello-WMF closed this task as Resolved.May 31 2022, 3:25 PM

Isaac mentioned this in T289532: Add more languages to Wikipedia Clickstream.Aug 29 2022, 5:39 PM