Change Details

We are currently starting to experiment with running jobs on Yarn via Skein in our Kubernetes Airflow instances (https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/887). Skein will need to `core-site.xml` and `hdfs-site.xml` configuration files to be deployed locally to be able to reach out to Hadoop, and we will need to have appropriate network policies to be put in place.

We are currently starting to experiment with running jobs on Yarn via Skein in our Kubernetes Airflow instances (https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/887). Skein will need to `core-site.xml`, `hdfs-site.xml` and `yarn-site.xml` configuration files to be deployed locally to be able to reach out to Hadoop, and we will need to have appropriate network policies to be put in place.

We are currently starting to experiment with running jobs on Yarn via Skein in our Kubernetes Airflow instances (https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/887). Skein will need to `core-site.xml`, `hdfs-site.xml` and `hdfsyarn-site.xml` configuration files to be deployed locally to be able to reach out to Hadoop, and we will need to have appropriate network policies to be put in place.