Refactor Existing Image Recommendation Code (may need breaking down to more tasks)
Closed, ResolvedPublic13 Estimated Story Points
Actions

Assigned To

None

Authored By

	lbowmaker
	Oct 7 2021, 2:00 PM

Description

User Story

As a platform engineer, I need to refactor the existing image recommendations code so that it can run efficiently in the available environment

Success Criteria

Refactored code uses Spark for CPU/memory intensive work
Refactored code does not store data locally but on HDFS
Code passes CI checks

TODO: Add any new success criteria from outstanding tasks

Related Objects
Search...

Status	Assigned	Task
Resolved	lbowmaker	T292218 Implement Automated Batch Execution for Non-Platform Engineering
Resolved	None	T292740 Refactor Existing Image Recommendation Code (may need breaking down to more tasks)
Resolved	None	T280585 📊Image Matching experiments should be deterministic

Event Timeline

lbowmaker created this task.Oct 7 2021, 2:00 PM

lbowmaker removed a project: Documentation.Oct 7 2021, 2:03 PM

lbowmaker moved this task from Backlog to Product Roadmap on the Generated Data Platform board.

lbowmaker reassigned this task from lbowmaker to • Clarakosi.Oct 13 2021, 3:33 PM

lbowmaker updated the task description. (Show Details)

lbowmaker set the point value for this task to 13.

lbowmaker moved this task from Product Roadmap to Ready/Groomed 📚 on the Generated Data Platform board.

• Clarakosi merged a task: T280585: 📊Image Matching experiments should be deterministic .Oct 13 2021, 4:36 PM

• Clarakosi updated the task description. (Show Details)

• Clarakosi added subscribers: gmodena, • Clarakosi.

• Clarakosi updated the task description. (Show Details)Oct 14 2021, 3:07 PM

• Clarakosi mentioned this in T276979: [PLACEHOLDER] 📊 Model, dataset versioning & metadata management.

• Clarakosi merged a task: T288289: Refactor Image Matching.Oct 19 2021, 1:31 PM

• Clarakosi added subscribers: fkaelin, FGoodwin, WDoranWMF.

• Clarakosi updated the task description. (Show Details)Oct 19 2021, 1:33 PM

• Clarakosi moved this task from Ready/Groomed 📚 to Work in Progress ⚙️ on the Generated Data Platform board.Oct 19 2021, 1:36 PM

• Clarakosi added a subtask: T280585: 📊Image Matching experiments should be deterministic .Nov 9 2021, 2:19 PM

• Clarakosi updated the task description. (Show Details)

I've updated the task to move the remaining work to T280585.

The current refactor changes are up for review at: https://github.com/mirrys/ImageMatching/pull/29

gmodena mentioned this in T292748: [SPIKE] Create Generic Components for Scheduling.Nov 16 2021, 4:01 PM

Has been merged into the ImageMatching repo on GitLab: https://gitlab.wikimedia.org/gmodena/ImageMatching/-/merge_requests/33

Tagging as Data Pipelines because it involves writing a dag. Untag if not relevant.

gmodena added a project: Data Pipelines.Mar 1 2022, 8:31 AM

Removing inactive task assignee

lbowmaker closed subtask T280585: 📊Image Matching experiments should be deterministic as Resolved.Apr 4 2022, 12:45 PM

lbowmaker closed this task as Resolved.Apr 4 2022, 12:50 PM

Refactor Existing Image Recommendation Code (may need breaking down to more tasks)Closed, ResolvedPublic13 Estimated Story PointsActions