Page MenuHomePhabricator

ores_predictions_daily DAG fails to overwrite files
Open, HighPublic

Description

The fetch_articletopic_prediction_thresholds and fetch_drafttopic_prediction_thresholds initially failed on june 16, 2022 for unknown reasons, and then continued to fail because the first execution left behind an empty destination file and the system refused to overwrite it. Ideally this should auto-magically fix itself and overwrite the invalid file. We already allow parquet datasets to be overwritten, we should be able to do soemthing similar here.