Page MenuHomePhabricator

Re-enable `set -e` in Wikibase entity dump scripts
Open, Needs TriagePublic

Description

We have repeatedly been slow to notice that the Wikidata entity dumps weren't generated correctly (see e.g. T403882). This is partially due to the fact that the dump generation scripts don't set -e, i.e. fail when one of the commands they execute fails. This in turn causes the airflow DAGs to report successful runs even though there were errors and the dump files aren't available.

This change was made during the migration to airflow (see T400383#11038714), but various problems have been fixed since then and they have been working fine in recent weeks. They should be robust enough for us to re-enable set -e so that we'll know if the dump generation fails.