Page MenuHomePhabricator
Create Task
Maniphest T305844

Migrate the projectview jobs
Closed, ResolvedPublic3 Estimated Story Points
Actions

Description

Goal:
Migrate the 2 ProjectView jobs to Airflow:

Job Details:

InputProcessingOutput
Hive TableHiveHive & Archive

Success Criteria:

  • Have the 2 Daily Jobs Migrated (SLA 6 Hours)
Gotchas
  • We already have an Airflow ArchiveOperator, but it might need adjustments!

Related Objects
Search...

Event Timeline

• EChetty created this task.Apr 11 2022, 1:18 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 11 2022, 1:18 PM
mforns added a project: Data-Engineering.May 4 2022, 1:10 PM
mforns updated the task description. (Show Details)
mforns added a parent task: T307500: Airflow Hackathon (May 2022).
mforns renamed this task from Migrate the ProjectView jobs to Airflow to Migrate the projectview jobs.May 4 2022, 1:56 PM
mforns moved this task from Incoming (new tickets) to Transform on the Data-Engineering board.May 11 2022, 6:45 PM
• EChetty assigned this task to Snwachukwu.May 23 2022, 3:44 PM
• EChetty moved this task from Discussed (Radar) to Estimated on the Data Pipelines board.
Snwachukwu added a subtask: T309023: Update and add copies of projectview hql script to analytics/refinery/hql path.May 24 2022, 12:57 PM
Snwachukwu added a project: Data-Engineering-Kanban.Jun 20 2022, 6:19 AM
Snwachukwu moved this task from Next Up to In Progress on the Data-Engineering-Kanban board.
• EChetty edited projects, added Data-Engineering-Planning; removed Data-Engineering.Jun 30 2022, 3:05 PM
JArguello-WMF edited projects, added Data-Engineering-Planning (Sprint 01); removed Data-Engineering-Planning.Jun 30 2022, 3:33 PM
JArguello-WMF moved this task from Ready to In progress on the Data-Engineering-Planning (Sprint 01) board.Jun 30 2022, 3:43 PM
• EChetty set the point value for this task to 3.Jun 30 2022, 5:19 PM
Snwachukwu moved this task from Estimated to In Review on the Data Pipelines board.Jul 5 2022, 3:55 PM
Snwachukwu moved this task from In progress to In code review on the Data-Engineering-Planning (Sprint 01) board.Jul 5 2022, 4:04 PM
Snwachukwu moved this task from In code review to In progress on the Data-Engineering-Planning (Sprint 01) board.Jul 6 2022, 4:39 PM
Snwachukwu moved this task from In Review to Estimated on the Data Pipelines board.
• EChetty moved this task from In progress to Paused/Blocked on the Data-Engineering-Planning (Sprint 01) board.Jul 7 2022, 4:37 PM
• EChetty moved this task from Paused/Blocked to In code review on the Data-Engineering-Planning (Sprint 01) board.Jul 13 2022, 4:06 PM
JArguello-WMF closed subtask T309023: Update and add copies of projectview hql script to analytics/refinery/hql path as Resolved.Jul 25 2022, 9:37 AM
• EChetty moved this task from Estimated to In Review on the Data Pipelines board.Jul 25 2022, 12:34 PM
• EChetty edited projects, added Data-Engineering-Planning (Sprint 02); removed Data-Engineering-Planning (Sprint 01).Jul 25 2022, 2:14 PM
• EChetty moved this task from Ready to In progress on the Data-Engineering-Planning (Sprint 02) board.Jul 25 2022, 3:29 PM
Snwachukwu moved this task from In progress to In code review on the Data-Engineering-Planning (Sprint 02) board.Jul 26 2022, 4:04 PM
JArguello-WMF moved this task from In Review to Sprint 00 on the Data Pipelines board.Jul 28 2022, 2:12 PM
JArguello-WMF edited projects, added Data Pipelines (Sprint 00); removed Data Pipelines.
• EChetty moved this task from In code review to Ready to deploy on the Data-Engineering-Planning (Sprint 02) board.Jul 28 2022, 4:07 PM
• EChetty removed a project: Data Pipelines (Sprint 00).Aug 1 2022, 3:19 PM
JArguello-WMF moved this task from Ready to deploy to In code review on the Data-Engineering-Planning (Sprint 02) board.Aug 1 2022, 3:19 PM
• EChetty moved this task from In code review to Paused/Blocked on the Data-Engineering-Planning (Sprint 02) board.Aug 2 2022, 4:08 PM
Snwachukwu moved this task from Paused/Blocked to Ready to deploy on the Data-Engineering-Planning (Sprint 02) board.Aug 3 2022, 4:07 PM
• EChetty moved this task from Ready to deploy to Done on the Data-Engineering-Planning (Sprint 02) board.Aug 11 2022, 4:04 PM
• EChetty closed this task as Resolved.Aug 13 2022, 2:13 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL