Maniphest T296531

Migrate the Medium Risk Refine Jobs to Airflow (50%)
Closed, DuplicatePublic
Actions

Assigned To

None

Authored By

	• EChetty
	Nov 26 2021, 1:48 PM

Tags

Referenced Files

None

Subscribers

Description

Goal: Migrate half Refine Jobs to Airflow:
Success Criteria:

Have the Jobs running on airflow with all Monitoring & Testing requirements met.
Job is sufficiently stable to remove it from the refine scheduler.
Ensure we have documented our process on how to view and modify the DAG

Tasks Remaining

Break out each job and estimate on each job link
Jobs: eventlogging_legacy, eventlogging_analytics

Event Timeline

• EChetty created this task.Nov 26 2021, 1:48 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 26 2021, 1:48 PM

• EChetty updated the task description. (Show Details)Nov 26 2021, 1:48 PM

• EChetty set the point value for this task to 9.Dec 13 2021, 5:15 PM

• EChetty updated the task description. (Show Details)Dec 13 2021, 5:26 PM

• EChetty moved this task from Backlog to Next Up (revisit every 2 sprints) on the Data Pipelines board.Jan 25 2022, 11:32 AM

• EChetty removed the point value for this task.Sep 26 2022, 2:29 PM

• EChetty moved this task from Next Up (revisit every 2 sprints) to Backlog on the Data Pipelines board.

JArguello-WMF added a project: Data Engineering and Event Platform Team.Jun 30 2023, 5:54 PM

Ottomata closed this task as a duplicate of T307505: Refine jobs should be scheduled by Airflow.Jul 6 2023, 1:57 PM