Maniphest T296527

Deploy DE Production Instance of Airflow
Closed, ResolvedPublic9 Estimated Story Points
Actions

Assigned To

Authored By

	• EChetty
	Nov 26 2021, 1:46 PM

Tags

Referenced Files

None

Subscribers

Description

Goal: Deploy Production version of Airflow

Success Criteria:

Have our Production instance of airflow available to the DE Team.
Check we have the DevOps, Monitoring and Evaluation tools we want/need in place.
Ensure we have documented our Development-Test-Deploy Path when creating and scheduling jobs

Tasks:

Spike Task: What do we need to know from M&E
Spike Task: Define Logging requirement
Documentation Requirements

Details

	Subject	Repo	Branch	Lines +/-
	airflow - Set SKEIN_CONFIG	operations/puppet	production	+8 -0

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		BTullis	T296527 Deploy DE Production Instance of Airflow
		Duplicate		None	T296535 Centralise DAG location and organisation
		Declined		None	T296536 Centralise DAG location and organisation
		Resolved		Ottomata	T296543 Tooling for Deploying Conda Environments
		Declined		None	T296544 Tooling for Deploying Gitrepos for Flat Files

Event Timeline

• EChetty created this task.Nov 26 2021, 1:46 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 26 2021, 1:46 PM

• EChetty renamed this task from Deploy DE Production version of Airflow to Deploy DE Production Instance of Airflow.Nov 26 2021, 1:55 PM

• EChetty updated the task description. (Show Details)Dec 13 2021, 5:01 PM

• EChetty set the point value for this task to 9.

• EChetty moved this task from Backlog to Next Up (revisit every 2 sprints) on the Data Pipelines board.Jan 25 2022, 11:32 AM

Change 760658 had a related patch set uploaded (by Ottomata; author: Ottomata):

[operations/puppet@production] airflow - Set SKEIN_CONFIG

https://gerrit.wikimedia.org/r/760658

gerritbot added a project: Patch-For-Review.Feb 7 2022, 9:03 PM

Change 760658 merged by Ottomata:

[operations/puppet@production] airflow - Set SKEIN_CONFIG

https://gerrit.wikimedia.org/r/760658

Maintenance_bot removed a project: Patch-For-Review.Feb 7 2022, 9:10 PM

Ottomata closed subtask T296543: Tooling for Deploying Conda Environments as Resolved.Mar 22 2022, 4:18 PM

• EChetty moved this task from Next Up (revisit every 2 sprints) to Epics on the Data Pipelines board.Sep 26 2022, 2:29 PM

• EChetty closed subtask T296536: Centralise DAG location and organisation as Declined.Nov 20 2022, 6:07 PM

I believe that we can now mark this ticket as resolved.