This task is to essentially port over the existing spark jobs from import_wikidata_ttl to the new data-reload Airflow DAG.
The goal is to get this running with Spark on Kubernetes vs on YARN, so we will need to coordinate with SRE and get their advice and help.
AC:
- RDF triples written to the new RDF tables in Ceph (side effect of the DAG)
- Files output to wikidata's CephFS volume with munged/split/skolemized data ready for the triplestore to index/ingest