Page MenuHomePhabricator

Investigate if timeouts during import are recoverable
Closed, DeclinedPublic

Description

If an import's database connection times out and subsequently errors out, the entire import will have to be restarted and the entire diff recalculated for the new run. It's unclear if this timeout is transitory (and a retry would keep the import going) or if the timeout is happening outside of the container's control. If we can recover instead of aborting/retrying on the next cron job, we'd save some time per import (~3 hours) whenever it happens.

IMO this is a nice to have but low priority. Recently, as of writing, quite a few timeouts occurred during sequential import but seems to have stabilized. If this persists, then this ticket should be prioritized accordingly. For context, the disconnects last week cost about 6-8 hours in recalculations but each import takes 24+ hours.

image.png (554×1 px, 68 KB)