Page MenuHomePhabricator

webrequest / webrequest raw quality check
Open, Needs TriagePublic

Description

Following the migration of webrequest load from Oozie to Airflow, we are not generating SUCCESS files anymore.

The systemd data quality check check_webrequest_partitions relied on those files. It was forgotten. So we have removed the alert by eliminating the job.

https://gerrit.wikimedia.org/r/c/operations/puppet/+/908529

Maybe we do need a data quality job for those datasets. Something that checks the hive partition exists and that it is populated with a minimum number (TBD) of rows.