Description

Currently it is possible to schedule dbt jobs: daily, weekly and monthly. With custom logical groups feature implemented, it'd be helpful to have the option to enable hourly schedule for custom DAG groups.

The immediate use case is api_requests_hourly_agg_daily which aggregates refined API requests to hourly granularity, but runs daily. This is a sizeable dataset, and will be better to have it run hourly to process one hour worth data at a time. Given frequent memory errors and also this significantly increasing the overall run time of dbt-daily, having an hourly schedule will help with resolving recurrent memory issues and also reduce risk for dbt-daily, as this can be moved out of the static DAG.

Acceptance Criteria

• Custom dbt logical groups can be scheduled hourly in Airflow.

Required

• Documentation?