GitLab backups were configured in T274463. After some tweaks they work reliably now. However during the implementation it was clear that we will reach scaling limits with the current solution. With more adoption and usage of GitLab, the key problems are:
* backups take quite a lot of space ("more" in comparison to Gerrit)
* backups need even more space during creation
* backup creation takes quite a lot of time
* backups are done every 24h
So long-term plan is needed how we want to do backups for GitLab. This design should include how the backups are created, backup frequency, storage location, rotation and if possible also monitoring of backups. Furthermore we need some estimates regarding disk usage and backup and restore times for future backup sizes.
Some ideas discussed in T274463 and other tasks which could influence the design:
* order hosts with bigger disks/get a second pair of disks (and tweak disk layout)
* Setup partial/incremental backups for GitLab T316935 / T324506
* use [alternative backup strategies](https://docs.gitlab.com/ee/raketasks/backup_restore.html#alternative-backup-strategies) for example bacula, rsync, pgbouncer/dump
* use alternative backup storage locations like a dedicated [s3 bucket/minio/cloud](https://docs.gitlab.com/ee/raketasks/backup_gitlab.html#upload-backups-to-a-remote-cloud-storage) or a [locally mounted share](https://docs.gitlab.com/ee/raketasks/backup_gitlab.html#upload-to-locally-mounted-shares)
* tbd