Page MenuHomePhabricator

[EPIC] Create airflow job to gather image recommendations and push to various persistence layers
Closed, ResolvedPublic

Description

NOTE: This work will need to be done in collaboration with the Data Platform Team, as it's their Generated Data Platform we'll be using

Now that we're pretty sure that pushing wikidata information into the weighted_tags field in the commons index improves image search on an experimental index, we need to do the same for the production commonswiki_file index

At the same time we also need to gather up all data relevant to image suggestions, and push it to various persistence layers for consumption by clients

Part 1

Part 2

Part 3

Part 4

Part 5

Related Objects

StatusSubtypeAssignedTask
ResolvedCBogen
Resolvedmfossati
Resolvedmfossati
Resolvedmfossati
DeclinedNone
ResolvedCparle
ResolvedCparle
ResolvedCparle
Resolvedmfossati
DeclinedCparle
Invalidmfossati
Resolvedmfossati
Resolvedmfossati
ResolvedCparle
Resolvedmfossati
ResolvedCparle
Resolvedmfossati
Resolvedmfossati
ResolvedCparle
Resolvedmfossati

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Cparle renamed this task from Create airflow job to inject wikidata information into the commons index to Create airflow job to inject wikidata information into the production commonswiki_file search index.Dec 1 2021, 9:26 AM
CBogen renamed this task from Create airflow job to inject wikidata information into the production commonswiki_file search index to [XL] Create airflow job to inject wikidata information into the production commonswiki_file search index.Dec 15 2021, 5:22 PM
Cparle updated the task description. (Show Details)
Cparle renamed this task from [XL] Create airflow job to inject wikidata information into the production commonswiki_file search index to Create airflow job to gather image recommendations and push to various persistence layers.Jan 21 2022, 6:06 PM
Cparle updated the task description. (Show Details)
CBogen renamed this task from Create airflow job to gather image recommendations and push to various persistence layers to [XL] Create airflow job to gather image recommendations and push to various persistence layers.Jan 26 2022, 5:27 PM

Moving to "blocked" until the other parts have completed.

CBogen renamed this task from [XL] Create airflow job to gather image recommendations and push to various persistence layers to [EPIC] Create airflow job to gather image recommendations and push to various persistence layers.Feb 23 2022, 5:31 PM
CBogen moved this task from Blocked to Epics on the Structured-Data-Backlog (Current Work) board.

Maybe we could close this now? All the parts are completed, anything outstanding is a bug and has its own ticket

mfossati claimed this task.