Page MenuHomePhabricator

The data check task in the image suggestions DAG is failing
Closed, ResolvedPublic

Description

The problem is the script doesn't use spark, but is called with SparkSubmitOperator. It needs to be called with SimpleSkeinOperator instead

Details

TitleReferenceAuthorSource BranchDest Branch
Use SimpleSkeinOperator instead of SparkSubmitOperator for data checkrepos/data-engineering/airflow-dags!406cparleT337410main
Customize query in GitLab