Spark SQL supports so-called initialization SQL files through a separate command-line argument -i path/to/init.sql. This feature is useful because it allows developers to put commonly used SQL code (like macros) in separate files, and then simply include them during execution.
Add support for this feature to our existing SparkSqlOperator.
amastilovic opened https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/2233
Implement -i init.sql feature of SparkSqlOperator
amastilovic closed https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/2233
Draft: Implement -i init.sql feature of SparkSqlOperator
Closing this task since the init file won't be very useful to us - SparkSQL doesn't support user-defined MACROs.