Maniphest T280799

[SPIKE] 📊Research how to run image matching on airflow
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• Clarakosi
	Apr 21 2021, 2:47 PM

Tags

Referenced Files

None

Subscribers

Description

Goal

Familiarize the team with airflow and uses within the analytics cluster

Acceptance Criteria

Evaluate similar user airflow DAG
- Setup airflow, run the pipeline and confirm dataset is comparable to production data
Design doc for how to implement airflow for image matching
Document process for running the pipeline for image matching

Notes:

Ignore airflow scheduler for now

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		• odimitrijevic	T282033 Airflow collaborations
		Resolved		• Clarakosi	T280799 [SPIKE] 📊Research how to run image matching on airflow

Event Timeline

• Clarakosi created this task.Apr 21 2021, 2:47 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 21 2021, 2:47 PM

• Clarakosi moved this task from Backlog to Ready on the Platform Team Workboards (Image Suggestion API) board.Apr 21 2021, 2:47 PM

• Clarakosi added a parent task: T282033: Airflow collaborations.May 5 2021, 6:44 PM

• Clarakosi claimed this task.May 11 2021, 2:42 PM

• Clarakosi moved this task from Ready to In progress on the Platform Team Workboards (Image Suggestion API) board.

• sdkim moved this task from In progress to Ready on the Platform Team Workboards (Image Suggestion API) board.Oct 13 2021, 2:25 PM

mforns subscribed.Nov 5 2021, 3:53 PM

@Clarakosi closing this as done, since the IMA PoC has been migrated to https://gitlab.wikimedia.org/repos/generated-data-platform/datapipelines

gmodena closed this task as Resolved.Mar 1 2022, 9:10 AM