Page MenuHomePhabricator
Create Task
Maniphest T300023

Low Risk Oozie Migration: wikidata_item_page_link
Closed, ResolvedPublic
Actions

Description

Goal:
Migrate the wikidata_item_page_link job to Airflow:

Job Details:

InputProcessingOutput
Hive TableSparkHive Table

Success Criteria:

  • Have the 1 Weekly Job Migrated (SLA 14 Days)

Details

SubjectRepoBranchLines +/-
Migrate wikidata/item_page_link/weeklyanalytics/refinery/sourcemaster+1 -0
Migrate wikidata/item_page_link/weekly from Oozie to Airflowanalytics/refinerymaster+176 -0
Migrate wikidata/item_page_link/weekly to Airflowanalytics/refinerymaster+154 -543
Customize query in gerrit

Related Objects
Search...

Event Timeline

• EChetty created this task.Jan 25 2022, 11:44 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 25 2022, 11:44 AM
• EChetty added a parent task: T299074: Migrate Oozie jobs to Airflow.Jan 25 2022, 12:01 PM
odimitrijevic moved this task from Incoming (new tickets) to Transform on the Data-Engineering board.Feb 1 2022, 11:11 PM
Antoine_Quhen claimed this task.Feb 3 2022, 2:28 PM
Antoine_Quhen moved this task from Backlog to Estimated on the Data Pipelines board.
Antoine_Quhen added a project: Data-Engineering-Kanban.Feb 7 2022, 5:30 PM
Antoine_Quhen moved this task from Next Up to In Progress on the Data-Engineering-Kanban board.
gerritbot added a comment.Feb 11 2022, 8:23 AM

Change 761876 had a related patch set uploaded (by Aqu; author: Aqu):

[analytics/refinery/source@master] Migrate wikidata/item_page_link/weekly

https://gerrit.wikimedia.org/r/761876

gerritbot added a project: Patch-For-Review.Feb 11 2022, 8:23 AM
gerritbot added a comment.Feb 12 2022, 3:46 PM

Change 762067 had a related patch set uploaded (by Aqu; author: Aqu):

[analytics/refinery@master] Migrate wikidata/item_page_link/weekly to Airflow

https://gerrit.wikimedia.org/r/762067

Antoine_Quhen moved this task from Estimated to In Review on the Data Pipelines board.Feb 14 2022, 4:36 PM
Antoine_Quhen moved this task from In Progress to In Code Review on the Data-Engineering-Kanban board.Feb 17 2022, 4:46 PM
gerritbot added a comment.Feb 21 2022, 2:46 PM

Change 762067 abandoned by Aqu:

[analytics/refinery@master] Migrate wikidata/item_page_link/weekly to Airflow

Reason:

Duplication of https://gerrit.wikimedia.org/r/q/I2a4fcdb8251aa2930424cd4de0b56eb1d348f2ca

https://gerrit.wikimedia.org/r/762067

Antoine_Quhen moved this task from Transform to Serve on the Data-Engineering board.Feb 25 2022, 4:06 PM
Antoine_Quhen moved this task from Serve to Transform on the Data-Engineering board.
Antoine_Quhen moved this task from In Code Review to Ready to Deploy on the Data-Engineering-Kanban board.Feb 28 2022, 4:10 PM
Antoine_Quhen moved this task from In Review to Done on the Data Pipelines board.Mar 8 2022, 4:20 PM
Antoine_Quhen moved this task from Ready to Deploy to Done on the Data-Engineering-Kanban board.
gerritbot added a comment.Apr 26 2022, 3:03 PM

Change 761876 merged by jenkins-bot:

[analytics/refinery/source@master] Migrate wikidata/item_page_link/weekly

https://gerrit.wikimedia.org/r/761876

Maintenance_bot removed a project: Patch-For-Review.Apr 26 2022, 3:31 PM
JArguello-WMF closed this task as Resolved.May 31 2022, 3:31 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL