Page MenuHomePhabricator

webrequest / webrequest raw quality check
Open, Needs TriagePublic


Following the migration of webrequest load from Oozie to Airflow, we are not generating SUCCESS files anymore.

The systemd data quality check check_webrequest_partitions relied on those files. It was forgotten. So we have removed the alert by eliminating the job.

Maybe we do need a data quality job for those datasets. Something that checks the hive partition exists and that it is populated with a minimum number (TBD) of rows.