Maniphest T309995

Migrate all Cassandra Jobs
Closed, ResolvedPublic3 Estimated Story Points
Actions

Assigned To

Authored By

	• EChetty
	Jun 6 2022, 3:15 PM

Tags

Referenced Files

None

Subscribers

MoritzMuehlenhoff

Description

Key Task

Migrate our Casandra Jobs to airflow.

Success Criteria

Have a Casandra Operator build and documented.
Migrate existing Cassandra Jobs.

notes:
Spark 3 means we don't need to build an operator

User Acceptance

Test that we can user the Spark 3 native query syntax to load into cassandra.

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		JAllemandou	T309995 Migrate all Cassandra Jobs
		Resolved		• NOkafor-WMF	T311507 Create cassandra loading HQL files from their oozie definition
		Resolved		• NOkafor-WMF	T311508 Create Airflow jobs using HQL files to load cassandra

Event Timeline

• EChetty created this task.Jun 6 2022, 3:15 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 6 2022, 3:15 PM

• EChetty added a project: Epic.Jun 6 2022, 3:18 PM

• EChetty assigned this task to JAllemandou.Jun 6 2022, 3:55 PM

• EChetty updated the task description. (Show Details)

• EChetty set the point value for this task to 3.

JAllemandou merged a task: T306962: Use airflow to load cassandra.Jul 5 2022, 4:19 PM

JAllemandou added subscribers: JAllemandou, MoritzMuehlenhoff, Antoine_Quhen and 2 others.

• EChetty added a project: Data-Engineering-Planning.Jul 25 2022, 12:34 PM

• EChetty moved this task from Backlog to Epics on the Data-Engineering-Planning board.Aug 1 2022, 1:07 PM

BTullis mentioned this in T313936: aqs1004 low disk space warning.Aug 18 2022, 9:12 AM

• EChetty closed subtask T311507: Create cassandra loading HQL files from their oozie definition as Resolved.Sep 26 2022, 2:37 PM

• EChetty closed subtask T311508: Create Airflow jobs using HQL files to load cassandra as Resolved.Oct 4 2022, 4:00 PM

Shall this be resolved @EChetty ?

• EChetty closed this task as Resolved.Nov 28 2022, 8:32 PM

• EChetty updated the task description. (Show Details)