Change Details

The first full-size set of XML/SQL dumps on Airflow is set for July 1st. It would be useful if we could pre-load the previous month's full dump onto the CephFS volume, so that the [[https://wikitech.wikimedia.org/wiki/Dumps/Rerunning_a_job#Prefetch_from_previous_dumps|prefetch mechanism]] will function.

The first full-size set of XML/SQL dumps on Airflow is set for July 1st. It would be useful if we could pre-load the previous month's full dump onto the CephFS volume, so that the [[https://wikitech.wikimedia.org/wiki/Dumps/Rerunning_a_job#Prefetch_from_previous_dumps|prefetch mechanism]] will function. There are 1008 wikis to be included and the full size of the set of dumps is 12 TB. ``` tullis@clouddumps1001:/srv/dumps/xmldatadumps/public$ find . -maxdepth 2 -name 20250601|wc -l 1008 btullis@clouddumps1001:/srv/dumps/xmldatadumps/public$ find . -maxdepth 2 -name 20250601 -exec du -ch {} + | grep total$ 12T total ```

The first full-size set of XML/SQL dumps on Airflow is set for July 1st. It would be useful if we could pre-load the previous month's full dump onto the CephFS volume, so that the [[https://wikitech.wikimedia.org/wiki/Dumps/Rerunning_a_job#Prefetch_from_previous_dumps|prefetch mechanism]] will function. There are 1008 wikis to be included and the full size of the set of dumps is 12 TB. ``` tullis@clouddumps1001:/srv/dumps/xmldatadumps/public$ find . -maxdepth 2 -name 20250601|wc -l 1008 btullis@clouddumps1001:/srv/dumps/xmldatadumps/public$ find . -maxdepth 2 -name 20250601 -exec du -ch {} + | grep total$ 12T total ```