Page MenuHomePhabricator
Create Task
Maniphest T377739

[Refine Refactoring] Refine Data Quality - late events, RefineMonitor refactor, etc.
Open, Needs TriagePublic
Actions

Description

STUB TICKET FOR NOW

We need to consider how we do alerting and data quality monitoring for Refine event tables after we do Refine on Airflow in T369845: [Refine Refactoring] Refine jobs should be scheduled by Airflow: deployment

RefineMonitor is scheduled by a systemd timer and uses the legacy RefineMonitor scheduling logic.

This is also related to late events monitoring (gobblin)

Related Objects
Search...

Event Timeline

Ottomata renamed this task from Refine Data Quality - late events, RefineMonitor refactor, etc. to [Refine Refactoring] Refine Data Quality - late events, RefineMonitor refactor, etc..Oct 21 2024, 2:48 PM
Ottomata created this task.
Ahoelzl edited projects, added Data-Engineering (Q2 2024 October 1st - December 31th); removed Data-Engineering, Data Pipelines.Oct 23 2024, 9:03 PM
JAllemandou awarded a token.Oct 29 2024, 10:46 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits