Tue, Nov 30
Wed, Nov 24
Linked ticket: https://phabricator.wikimedia.org/T281517
Tue, Nov 16
Subscribed DE - please add any comments/questions/suggestions.
Mon, Nov 15
Wed, Nov 10
Does this require data persistence of the suggestions or are they done on the fly?
Tue, Nov 9
First draft of process:
@gmodena - I think it would be useful to generate an airflow DAG skeleton too, at least a simple example of how to execute the data pipeline code.
Created this placeholder for our backlog in case there are any comments that require further work:
Thu, Nov 4
Data Pipeline (Lives in project name in repo) Checks:
Nov 3 2021
Nov 1 2021
@kostajh - what would happen if a page title changes between the image rec output and someone viewing the image rec then calling the API?
Oct 27 2021
Adding some comments from the grooming session today:
Oct 26 2021
Let's say a dataset producer only cares about image recs and has no involvement in similar users.
@gmodena - my thoughts for this task was to do something simple to also support a basic use case for a dataset producer/platform engineer. For example, 'I expected my Airflow DAG to generate a file, I can't see the file so I'll check for any errors in Airflow and the UI will show me the stack trace'.
Oct 21 2021
@Clarakosi @gmodena - Are you able to answer the first question above? Seems like there isn't a good reason to store page title? Not sure if there was any reasoning to it in the original requirements.
Oct 20 2021
Some additional comments on the database side after discussion with SRE and Platform team today: