The data reload pipeline uses a topolvm-backed PVC to store data across tasks. The plan (T428235) is that at the beginning of the pipeline we request a PV, use it during the DAG to create the index, and then destroy it at the end. We need to validate that topolvm works well enough for this storage.

In an earlier iteration of the deployment we used network attached storage (cephFS) and the fallback, if topolvm proves completely unsuitable, would be to use that.

Earlier work explored using emptyDir local storage in the deployment, but this was rejected in favor of network attached storage. And the emptyDir solution is not viable for the pipeline, because it only lives as long as the pod.

We should validate that TopoLVM works for us and runs in a time we're happy with. The target can be 5 hours, which is how long it takes to run the qlever indexer on wdqs1030. For time efficiency, use a smaller index when appropriate.

For the indexing pipeline we need to measure runtimes with different core/memory resource allocation. (What is the lower bound where adding resources doesn't help?) Based on this, determine the optimal resource allocation.

Explore also whether thin provisioning (on-demand space allocation) of topolvm makes a difference.