Page MenuHomePhabricator
Create Task
Maniphest T301352

Create DAG for Snapshots service
Closed, ResolvedPublic8 Estimated Story Points
Actions

Description

Create DAG to trigger Export endpoint in Scheduler service every day so our customers can download latest exports every day and we can monitor task execution and setup retries.

Acceptance criteria
DAG for daily snapshots is created and activated on dev environement.

To-Do

  • create DAG for snapshots that will run export on all projects daily in services/scheduler repository

Notes
Refer to other DAG(s) in the scheduler as an example.

Related Objects
Search...

Event Timeline

Protsack.stephan created this task.Feb 9 2022, 2:20 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 9 2022, 2:20 PM
Protsack.stephan triaged this task as Medium priority.Feb 9 2022, 2:21 PM
Protsack.stephan lowered the priority of this task from Medium to Low.
Protsack.stephan added a subtask: T301353: Add TLS to Snapshots service related DAG.
Protsack.stephan added a parent task: T301347: Initial Setup For Exports Repository.
Protsack.stephan set the point value for this task to 8.May 4 2022, 3:18 PM
Protsack.stephan moved this task from Incoming to To Be Estimated/To Be Discussed on the Wikimedia Enterprise board.Oct 11 2022, 7:28 PM
Protsack.stephan renamed this task from Create DAG For Exports Service to Create DAG for snapshots service.Oct 12 2022, 9:38 AM
Protsack.stephan renamed this task from Create DAG for snapshots service to Create DAG for Snapshots service.
Protsack.stephan raised the priority of this task from Low to Needs Triage.
Protsack.stephan added a project: Wikimedia Enterprise Engineering.Oct 12 2022, 9:41 AM
Protsack.stephan updated the task description. (Show Details)Oct 12 2022, 1:45 PM
Protsack.stephan removed the point value 8 for this task.
Protsack.stephan set the point value for this task to 8.Nov 23 2022, 4:00 PM
Protsack.stephan moved this task from To Be Estimated/To Be Discussed to Estimated /Discussed on the Wikimedia Enterprise board.Dec 1 2022, 9:11 AM
Protsack.stephan closed subtask T301353: Add TLS to Snapshots service related DAG as Invalid.Dec 1 2022, 9:49 AM
Protsack.stephan triaged this task as High priority.Dec 1 2022, 10:27 AM
REsquito-WMF changed the task status from Open to In Progress.Dec 8 2022, 1:19 PM
REsquito-WMF claimed this task.
REsquito-WMF moved this task from Estimated /Discussed to In Progress on the Wikimedia Enterprise board.
REsquito-WMF moved this task from In Progress to Merge Request on the Wikimedia Enterprise board.Dec 12 2022, 12:23 PM
REsquito-WMF moved this task from Merge Request to Blocked on the Wikimedia Enterprise board.EditedDec 19 2022, 1:56 PM

Blocked, needs api built to call it from airflow.

REsquito-WMF added a comment.Dec 19 2022, 2:06 PM

https://gitlab.gluzdov.com/wikimedia-enterprise/services/scheduler/-/merge_requests/7

Protsack.stephan moved this task from Blocked to Machine Readability PB on the Wikimedia Enterprise board.Dec 22 2022, 1:40 PM
Protsack.stephan moved this task from Machine Readability PB to Done 31 on the Wikimedia Enterprise board.Dec 26 2022, 1:20 PM
Felixejofre moved this task from Done 31 to Done Sprint 30(Dec01-Jan11 2022) on the Wikimedia Enterprise board.Dec 29 2022, 1:04 PM
Daria_Kevana changed the task status from In Progress to Open.Jan 3 2023, 3:54 PM
JArguello-WMF moved this task from Done Sprint 30(Dec01-Jan11 2022) to Done 31 on the Wikimedia Enterprise board.Jul 14 2023, 3:32 PM
JArguello-WMF closed this task as Resolved.Jul 14 2023, 3:35 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits