Page MenuHomePhabricator
Create Task
Maniphest T328933

[Airflow] Migrate 3 Druid jobs
Closed, ResolvedPublic
Actions

Description

Migrate 3 Druid loading jobs for event-based datasets to Airflow.
editattemptstep
navigationtiming
prefupdate

Details

SubjectRepoBranchLines +/-
analytics::refinery::job::druid_load.pp: remove absented jobsoperations/puppetproduction+0 -34
analytics::refinery::job::druid_load.pp: Absent 3 jobs to migrateoperations/puppetproduction+3 -0
Customize query in gerrit

Event Timeline

mforns created this task.Feb 6 2023, 3:59 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 6 2023, 3:59 PM
mforns claimed this task.Feb 6 2023, 3:59 PM

Here's the related merge request:
https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/212

mforns moved this task from Ready to Ready to Deploy on the Data Pipelines (Sprint 07) board.Feb 6 2023, 3:59 PM
• EChetty edited projects, added Data Pipelines (Sprint 08); removed Data Pipelines (Sprint 07).Feb 7 2023, 12:22 PM
• EChetty moved this task from Ready to Ready to Deploy on the Data Pipelines (Sprint 08) board.Feb 7 2023, 12:23 PM
gerritbot added a comment.Feb 9 2023, 2:40 PM

Change 888018 had a related patch set uploaded (by Mforns; author: Mforns):

[operations/puppet@production] analytics::refinery::job::druid_load.pp: Absent 3 jobs to migrate

https://gerrit.wikimedia.org/r/888018

gerritbot added a project: Patch-For-Review.Feb 9 2023, 2:40 PM
gerritbot added a comment.Feb 9 2023, 3:52 PM

Change 888018 merged by Stevemunene:

[operations/puppet@production] analytics::refinery::job::druid_load.pp: Absent 3 jobs to migrate

https://gerrit.wikimedia.org/r/888018

Maintenance_bot removed a project: Patch-For-Review.Feb 9 2023, 4:30 PM
mforns added a comment.Feb 9 2023, 7:20 PM

The 3 jobs are in Airflow now!

mforns moved this task from Ready to Deploy to Done on the Data Pipelines (Sprint 08) board.Feb 9 2023, 7:20 PM
gerritbot added a comment.Feb 9 2023, 8:20 PM

Change 888082 had a related patch set uploaded (by Mforns; author: Mforns):

[operations/puppet@production] analytics::refinery::job::druid_load.pp: remove absented jobs

https://gerrit.wikimedia.org/r/888082

gerritbot added a project: Patch-For-Review.Feb 9 2023, 8:20 PM
gerritbot added a comment.Feb 10 2023, 1:28 PM

Change 888082 merged by Stevemunene:

[operations/puppet@production] analytics::refinery::job::druid_load.pp: remove absented jobs

https://gerrit.wikimedia.org/r/888082

Maintenance_bot removed a project: Patch-For-Review.Feb 10 2023, 1:30 PM
JArguello-WMF closed this task as Resolved.Feb 17 2023, 7:14 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits